feat: measure configurable synthetic pilot workload in isolated PostgreSQL runtime

Assistant: codex
Assistant-Model: gpt-6-astra
Assistant-Session: 01a15374-f59f-7a31-86d2-d23b5d2ed181
This commit is contained in:
tegwick 2026-10-19 14:54:19 +02:00
parent a1793dbe12
commit 11f7c8ae83
12 changed files with 1134 additions and 0 deletions

View file

@ -189,3 +189,16 @@ existing bytes remain available for offline session startup; refreshing it does
not require a Git commit. Authored workplans and the generated WORK-RECORDS index
retain their existing source and staged-generation contracts. No runtime or
credential change follows from this repository hygiene policy.
## Provisional synthetic pilot sizing
`scripts/synthetic_pilot.py` drives real loopback HTTP through an isolated Gunicorn
instance and disposable PostgreSQL16. `config/synthetic-pilot.json` tunes workload
assumptions, and `config/synthetic-sizing-margins.json` tunes the conservative local
resource-envelope calculation. Per-operation latency/errors and separate document
transfer, process CPU/RSS and database snapshots are retained. The 2026-10-19 local
sample passes the two-user ordinary p95<=2s/zero-error criterion; it does not establish
human acceptance or the production60m CPU request. Coverage and provisional values
are in `docs/synthetic-pilot.md` and `docs/evidence/2026-10-19-synthetic-sizing/`.
No deployment or publication is performed by these tools.

View file

@ -0,0 +1,12 @@
{
"users": 2,
"workers": 2,
"iterations": 100000,
"tenders": 100,
"document_bytes": 1048576,
"think_seconds": 0.2,
"timeout_seconds": 30,
"ordinary_p95_seconds": 2.0,
"duration_seconds": 180,
"idle_seconds": 30
}

View file

@ -0,0 +1,6 @@
{
"cpu_request_multiplier": 1.5,
"cpu_limit_multiplier": 1.5,
"memory_request_multiplier": 1.25,
"memory_limit_multiplier": 1.5
}

View file

@ -0,0 +1,44 @@
# Isolated synthetic pilot sizing — 2026-10-19
The user authorized provisional synthetic sizing with adjustable assumptions.
Run locally against fresh PostgreSQL16 and loopback Gunicorn, not production.
Two users, two workers,100 initial tenders,1MiB documents,0.2-second think time.
30s idle then180.889s monotonic ordinary duration;378 complete cycles.
| Result | Value |
| --- | --- |
| Ordinary requests / errors / p95 | 2272 / 0 / 0.226793s |
| Document transfer requests / errors / p95 | 756 / 0 / 0.095466s |
| Aggregate app CPU p95 / observed peak | 1298.30m / 1916.96m |
| Peak summed app RSS | 193.84MiB |
| Process samples / database interval snapshots | 422 / 103 |
| Conservative local CPU request / limit candidate | 1950m / 2900m |
| Conservative local memory request / limit candidate | 256Mi / 320Mi |
This ~16.7-requests/second synthetic mix is deliberately aggressive. It is not
measured human demand. This unbounded local envelope exceeds the accepted production1000m CPU limit
and the current395m reservation residual reported by the allocation owner. It is
not a recommendation to raise that limit. The next comparison is an isolated
application-container run at the existing1000m/1Gi limit;60m is a request, not a quota.
The candidate envelope derives from configurable margins
in `config/synthetic-sizing-margins.json`; no deployment change is made or implied.
The eight-logical-CPU Ryzen5 PRO3500U host had no harness app CPU quota. Neither
these results nor the passing latency criterion prove the existing60m production
request. Database snapshots are retained separately and not charged to app demand.
RSS sums shared resident pages more than once; per-worker CPU is aggregate/2,
not independently observed worker load. Idle is measured; startup timing, restart,
editing, separate bursts, browser/ingress/SSO and production quota/throttling are not.
Repeat with revised assumptions or approved target-runtime constraints before
production right-sizing. Human pilot acceptance remains open.
`sizing-run.json.gz` preserves raw per-operation and resource samples, source/config/
image digests and hardware. `sizing-envelope.json` is reproducible with the summary
command in `docs/synthetic-pilot.md`. `executed-sizing-driver.py` is the exact run
snapshot matching its retained digest; the maintained script subsequently received
only import/format lint cleanup. `exploratory.json.gz` retains the earlier15.7s
sample, not used as the sizing recommendation. Use monotonic duration for rates;
UTC boundaries are retained as observed and may differ due to wall-clock adjustment.
Validation: six focused configuration/percentile/sizing tests; Ruff clean for
maintained scripts; complete real HTTP run passed all3028 operations. No customer
records, external identity, live credentials or production writes were used.

View file

@ -0,0 +1,292 @@
#!/usr/bin/env python3
"""Provisional loopback HTTP workload using Gunicorn and disposable PostgreSQL."""
import argparse
import atexit
import concurrent.futures
import hashlib
import json
import math
import os
from pathlib import Path
import platform
import socket
import subprocess
import urllib.request
import urllib.parse
import urllib.error
import http.cookiejar
import uuid
import sys
import tempfile
import threading
import time
from datetime import datetime, timezone
def summarize(rows):
durations = sorted(row['seconds'] for row in rows)
return {'requests': len(rows), 'errors': sum(not r['ok'] for r in rows),
'p95_seconds': durations[math.ceil(len(durations) * .95) - 1]
if durations else None}
def validate(config):
for key in ('users', 'iterations', 'tenders', 'document_bytes', 'workers', 'timeout_seconds'):
if type(config[key]) is not int or config[key] < 1:
raise ValueError(f'{key} must be a positive integer')
for key in ('think_seconds', 'ordinary_p95_seconds', 'duration_seconds', 'idle_seconds'):
if not isinstance(config[key], (int, float)) or not math.isfinite(config[key]):
raise ValueError(f'{key} must be finite')
if config['think_seconds'] < 0 or config['ordinary_p95_seconds'] <= 0:
raise ValueError('invalid timing parameter')
if config['duration_seconds'] <= 0 or config['idle_seconds'] < 0:
raise ValueError('duration must be positive and idle nonnegative')
if config['document_bytes'] < 8:
raise ValueError('document_bytes must be at least 8')
return config
def run(config, root):
# Never consume a deployment database, media tree, session or credential.
os.environ['DATABASE_URL'] = os.environ['SYNTHETIC_DATABASE_URL']
os.environ['SECRET_KEY'] = 'synthetic-disposable-not-a-deployment-key'
os.environ['DJANGO_SETTINGS_MODULE'] = 'synthetic_settings'
(root / 'synthetic_settings.py').write_text(
'from vergabe_teilnahme.settings.base import *\n'
'DEBUG = False\nALLOWED_HOSTS = ["127.0.0.1"]\nNETKINGDOM_ENABLED = False\n'
f'MEDIA_ROOT = {str(root / "media")!r}\n'
f'ISSUE_FACADE_LOCAL_DB = {str(root / "issues.db")!r}\n'
'ISSUE_FACADE_GITEA = None\nFORCE_SCRIPT_NAME = None\nMEDIA_URL = "/media/"\n'
'EMAIL_BACKEND = "django.core.mail.backends.locmem.EmailBackend"\n'
'SESSION_COOKIE_SECURE = False\nCSRF_COOKIE_SECURE = False\n'
)
sys.path.insert(0, str(root))
import django
django.setup()
from django.core.management import call_command
from django.db import connections, connection
from vergabe_teilnahme.apps.accounts.models import Mitarbeiter
from vergabe_teilnahme.apps.ausschreibungen.models import Ausschreibung
from vergabe_teilnahme.apps.dokumente.models import Dokument
call_command('migrate', verbosity=0)
for i in range(config['tenders']):
Ausschreibung.objects.create(titel=f'Synthetic tender {i}', ausschreiber='Synthetic')
users = []
password = uuid.uuid4().hex
for i in range(config['users']):
user = Mitarbeiter.objects.create_user(username=f'synthetic-{i}', password=password)
users.append(user.username)
with socket.socket() as sock:
sock.bind(('127.0.0.1', 0))
port = sock.getsockname()[1]
base_url = f'http://127.0.0.1:{port}'
env = dict(os.environ, PYTHONPATH=str(root) + os.pathsep + str(Path.cwd()))
server_log = (root / 'server.log').open('w')
server = subprocess.Popen([sys.executable, '-m', 'gunicorn',
'vergabe_teilnahme.wsgi:application', '--bind', f'127.0.0.1:{port}',
'--workers', str(config['workers']), '--timeout', '30'], env=env,
stdout=server_log, stderr=server_log)
atexit.register(lambda: server.terminate() if server.poll() is None else None)
for _ in range(100):
try:
with urllib.request.urlopen(base_url + '/health/', timeout=1) as response:
if response.status == 200:
break
except OSError:
if server.poll() is not None:
raise RuntimeError('Gunicorn startup failed; see isolated server log')
time.sleep(.1)
else:
server.terminate()
raise RuntimeError('Gunicorn readiness timeout')
samples = []
db_samples = []
sampling_done = threading.Event()
phase = {'name': 'idle'}
def sample():
ticks_per_second = os.sysconf('SC_CLK_TCK')
while not sampling_done.is_set():
pids = [server.pid]
try:
pids += [int(x) for x in Path(f'/proc/{server.pid}/task/{server.pid}/children').read_text().split()]
cpu, rss = 0, 0
for pid in pids:
fields = Path(f'/proc/{pid}/stat').read_text().split(') ', 1)[1].split()
cpu += (int(fields[11]) + int(fields[12])) / ticks_per_second
rss += int(fields[21]) * os.sysconf('SC_PAGE_SIZE')
samples.append({'at': datetime.now(timezone.utc).isoformat(),
'monotonic_seconds': time.monotonic(), 'phase': phase['name'],
'cpu_seconds': cpu, 'rss_bytes': rss, 'pids': pids})
except (OSError, ValueError):
samples.append({'at': datetime.now(timezone.utc).isoformat(), 'sample_error': True})
sampling_done.wait(.5)
def sample_db():
while not sampling_done.is_set():
value = subprocess.check_output(['docker', 'stats', '--no-stream', '--format',
'{{json .}}', os.environ['SYNTHETIC_CONTAINER']], text=True)
db_samples.append({'at': datetime.now(timezone.utc).isoformat(),
'phase': phase['name'], 'docker_stats': json.loads(value)})
sampling_done.wait(1)
sampler = threading.Thread(target=sample, daemon=True)
db_sampler = threading.Thread(target=sample_db, daemon=True)
sampler.start()
db_sampler.start()
time.sleep(config['idle_seconds'])
phase['name'] = 'ordinary'
deadline = time.monotonic() + config['duration_seconds']
completed = [0] * config['users']
barrier = threading.Barrier(config['users'])
def worker(i):
jar = http.cookiejar.CookieJar()
class NoRedirect(urllib.request.HTTPRedirectHandler):
def redirect_request(self, *args, **kwargs):
return None
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(jar), NoRedirect())
rows = []
def request(operation, method, path, data=None, expected=200, transfer=False):
started = time.perf_counter()
try:
headers = {}
body = None
if method == 'post':
token = next((c.value for c in jar if c.name == 'csrftoken'), '')
fields = dict(data or {}, csrfmiddlewaretoken=token)
if transfer:
boundary = 'synthetic-' + uuid.uuid4().hex
body = b''
for name, value in fields.items():
body += f'--{boundary}\r\nContent-Disposition: form-data; name="{name}"\r\n\r\n{value}\r\n'.encode()
body += (f'--{boundary}\r\nContent-Disposition: form-data; name="datei"; filename="synthetic.pdf"\r\nContent-Type: application/pdf\r\n\r\n'.encode()
+ b'%PDF-1.4' + b'x' * (config['document_bytes'] - 8)
+ f'\r\n--{boundary}--\r\n'.encode())
headers['Content-Type'] = f'multipart/form-data; boundary={boundary}'
else:
body = urllib.parse.urlencode(fields).encode()
headers['Referer'] = base_url + path
req = urllib.request.Request(base_url + path, data=body, headers=headers,
method=method.upper())
try:
response = opener.open(req, timeout=config['timeout_seconds'])
except urllib.error.HTTPError as exc:
response = exc
received = len(response.read())
status = response.status
ok = status == expected
response.close()
except Exception:
status, ok, received = None, False, 0
rows.append({'user': i, 'operation': operation, 'transfer': transfer,
'seconds': time.perf_counter() - started, 'status': status,
'ok': ok, 'response_bytes': received})
return ok
request('login-form', 'get', '/accounts/login/')
barrier.wait(timeout=30)
request('login', 'post', '/accounts/login/',
{'username': users[i], 'password': password}, 302)
for n in range(config['iterations']):
if time.monotonic() >= deadline:
break
request('list', 'get', '/ausschreibungen/')
request('search', 'get', '/ausschreibungen/?q=Synthetic')
title = f'Workload-{i}-{n}'
created = request('tender-create', 'post', '/ausschreibungen/neu/',
{'titel': title, 'ausschreiber': 'Synthetic'}, 302)
tender = Ausschreibung.objects.filter(titel=title).first() if created else None
if tender:
base = f'/ausschreibungen/{tender.pk}/'
request('detail', 'get', base)
request('lot-create', 'post', base + 'lose/neu/',
{'losnummer': '1', 'lostitel': 'Synthetic lot'}, 302)
request('task-create', 'post', base + 'aufgaben/neu/',
{'titel': 'Synthetic task', 'typ': 'fachlich', 'prioritaet': 2}, 302)
request('document-upload', 'post', base + 'dokumente/hochladen/',
{'kategorie': 'intern',
'version': '1.0'}, 302, True)
document = Dokument.objects.filter(ausschreibung=tender).first()
if document:
request('document-download', 'get', document.datei.url, transfer=True)
completed[i] += 1
time.sleep(config['think_seconds'])
connections.close_all()
return rows
started = datetime.now(timezone.utc).isoformat()
before = time.perf_counter()
with concurrent.futures.ThreadPoolExecutor(max_workers=config['users']) as pool:
rows = [r for group in pool.map(worker, range(config['users'])) for r in group]
ordinary = summarize([r for r in rows if not r['transfer']])
transfers = summarize([r for r in rows if r['transfer']])
complete = all(completed) and len(rows) == sum(completed) * 8 + config['users'] * 2
result = {'schema_version': 1, 'started_at': started,
'finished_at': datetime.now(timezone.utc).isoformat(),
'duration_seconds': time.perf_counter() - before, 'config': config,
'environment': {'python': platform.python_version(), 'django': django.get_version(),
'platform': platform.platform(), 'database': 'disposable PostgreSQL 16 container',
'transport': 'loopback HTTP Gunicorn',
'workers': config['workers']},
'ordinary': ordinary, 'document_transfer': transfers, 'complete': complete,
'provisional_pass': complete and ordinary['errors'] == 0
and transfers['errors'] == 0
and ordinary['p95_seconds'] <= config['ordinary_p95_seconds'],
'limitations': ['No browser, ingress, TLS, SSO or application container quota',
'No CPU/memory allocation recommendation or human acceptance',
'No timed startup/restart or distinct burst; configured idle and ordinary phases only'],
'operations': rows, 'runtime_samples': samples, 'database_samples': db_samples,
'completed_iterations': completed}
sampling_done.set()
sampler.join()
db_sampler.join()
server.terminate()
server.wait(timeout=10)
server_log.close()
connections.close_all()
return result
if __name__ == '__main__':
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--config', type=Path, required=True)
parser.add_argument('--output', type=Path, required=True)
args = parser.parse_args()
config = validate(json.loads(args.config.read_text()))
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
name = 'vergabe-synthetic-' + uuid.uuid4().hex[:12]
# Cached image only, loopback random host port, no existing volume or database.
subprocess.run(['docker', 'run', '--detach', '--pull=never', '--name', name,
'--publish', '127.0.0.1::5432', '--env', 'POSTGRES_DB=synthetic',
'--env', 'POSTGRES_USER=synthetic', '--env', 'POSTGRES_PASSWORD=synthetic-local-only',
'postgres:16-alpine'], check=True, stdout=subprocess.DEVNULL)
try:
for _ in range(100):
ready = subprocess.run(['docker', 'exec', name, 'pg_isready', '-U', 'synthetic'],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
if ready.returncode == 0:
break
time.sleep(.1)
else:
raise RuntimeError('isolated PostgreSQL startup failed')
port = subprocess.check_output(['docker', 'port', name, '5432/tcp'], text=True).strip().split(':')[-1]
os.environ['SYNTHETIC_CONTAINER'] = name
os.environ['SYNTHETIC_DATABASE_URL'] = f'postgres://synthetic:synthetic-local-only@127.0.0.1:{port}/synthetic'
with tempfile.TemporaryDirectory(prefix='vergabe-synthetic-') as directory:
result = run(config, Path(directory))
result['postgres_image_id'] = subprocess.check_output(
['docker', 'inspect', '--format', '{{.Image}}', name], text=True).strip()
result['postgres_final_stats'] = json.loads(subprocess.check_output(
['docker', 'stats', '--no-stream', '--format', '{{json .}}', name], text=True))
finally:
subprocess.run(['docker', 'rm', '--force', '--volumes', name], check=True,
stdout=subprocess.DEVNULL)
result['source_revision'] = subprocess.check_output(['git', 'rev-parse', 'HEAD'], text=True).strip()
result['driver_sha256'] = hashlib.sha256(Path(__file__).read_bytes()).hexdigest()
app_files = sorted(Path('vergabe_teilnahme').rglob('*.py')) + [Path('uv.lock'), Path('pyproject.toml')]
result['app_source_sha256'] = hashlib.sha256(b''.join(str(p).encode() + b'\0' + p.read_bytes() for p in app_files)).hexdigest()
result['host'] = {'logical_cpus': os.cpu_count(), 'affinity_cpus': len(os.sched_getaffinity(0)),
'cpu_model': next((x.split(':', 1)[1].strip() for x in Path('/proc/cpuinfo').read_text().splitlines() if x.startswith('model name')), 'unknown'),
'memory_total': next(x for x in Path('/proc/meminfo').read_text().splitlines() if x.startswith('MemTotal:')),
'cgroup_cpu_max': Path('/sys/fs/cgroup/cpu.max').read_text().strip() if Path('/sys/fs/cgroup/cpu.max').exists() else 'unavailable',
'app_cpu_quota': 'not imposed by harness'}
result['config_sha256'] = hashlib.sha256(args.config.read_bytes()).hexdigest()
args.output.write_text(json.dumps(result, indent=2) + '\n')
print(json.dumps({k: result[k] for k in ('ordinary', 'document_transfer', 'complete',
'provisional_pass')}))
raise SystemExit(0 if result['provisional_pass'] else 1)

View file

@ -0,0 +1,30 @@
{
"status": "provisional-local-host-only",
"application_observed": {
"cpu_p95_m": 1298.300558321979,
"cpu_peak_m": 1916.960754568632,
"rss_sum_peak_mib": 193.8359375,
"sample_intervals": 361,
"normalized_cpu_p95_m_per_worker": 649.1502791609895
},
"suggested_application": {
"cpu_request_m": 1950,
"cpu_limit_m": 2900,
"memory_request_mib": 256,
"memory_limit_mib": 320
},
"margins": {
"cpu_request_multiplier": 1.5,
"cpu_limit_multiplier": 1.5,
"memory_request_multiplier": 1.25,
"memory_limit_multiplier": 1.5
},
"limits": [
"Per-worker number is normalized aggregate, not an independently sampled worker.",
"RSS sum double-counts shared pages; conservative upper bound, not unique memory.",
"Observed local CPU is hardware/think-time dependent; quota effects untested.",
"Database/container demand is separate and not included in application request.",
"No production apply or human acceptance; validate target runtime before right-sizing."
],
"revisit": "Changed concurrency, corpus, document size, workers, image, hardware or quota; then repeat under approved target runtime including bursts and restart."
}

82
docs/synthetic-pilot.md Normal file
View file

@ -0,0 +1,82 @@
# Provisional synthetic pilot workload
The owner implementation is `scripts/synthetic_pilot.py`, configured by
`config/synthetic-pilot.json`. It reuses the application routes/forms exercised
by existing workflow tests. There was no existing performance driver in this
repository or railiance-apps. The driver uses real loopback HTTP, cookie sessions,
CSRF, Gunicorn and a fresh PostgreSQL 16 container; it never points to an existing
service/database. No live credentials, customer records or publication are needed.
Run from the application repository with its installed locked environment and
cached `postgres:16-alpine` Docker image:
```sh
.venv/bin/python scripts/test_synthetic_pilot.py
.venv/bin/python scripts/synthetic_pilot.py \
--config config/synthetic-pilot.json --output /tmp/vergabe-sizing.json
```
The container uses a unique name, random loopback database port and disposable
volume; cleanup removes only that container and its volume. Gunicorn binds a
random loopback port. Database, media and issue storage are isolated; temporary
passwords exist only for generated local users. The local settings disable SSO;
this measures local password login, not the invited deployment's identity flow.
No `seed_dev` command or existing PostgreSQL volume is reused.
## Adjustable assumptions
Defaults are two users, two synchronous Gunicorn workers, 100 initial tenders,
180 seconds of ordinary work after 30 seconds idle, and 0.2 seconds think time
per cycle. Each cycle creates a tender, lot and task, and reads list, search and
detail pages; it uploads then downloads one 1 MiB synthetic PDF. Thus the corpus
grows during the run. This is an explicit provisional operation mix, not observed
customer demand. `iterations` is a per-user safety ceiling; duration normally
ends the run first. User/worker count, duration, idle, iteration ceiling, corpus,
think time, document size, HTTP timeout and ordinary p95 threshold are tunable.
The PDF bytes are a transfer fixture, not a claim of document parser fidelity.
Pass requires all expected operations, ordinary p95 <= 2 seconds and zero failed
ordinary/transfer requests. Redirects are not automatically followed: rejected
login redirects cannot count as successful data-page responses. Document transfer
samples are reported separately. Ordinary samples include the login page/login;
fixture creation/migration and readiness are outside the measured workload.
## Coverage and interpretation
The result retains individual status/latency/response-size records, phase-tagged
0.5-second Gunicorn process CPU-time/RSS samples, separate interval Docker database
stats, driver/config/application source digests, runtime versions, image identity,
hardware and UTC boundaries. App RSS sums process resident sets and can double-count
shared pages; CPU reflects this host and has no app quota. Database snapshots are
separate demand, not allocated to the app by inference. A shared deployment database
requires a baseline and attribution before using these observations for capacity.
Startup is readiness-checked, not timed. Idle and fixed-think-time ordinary phases
are measured; restart and a distinct burst phase are not. Ingress/TLS/browser/SSO,
production quotas, throttling and shared services are unmeasured. Local success does
not establish human acceptance or prove the existing 60m production request.
Revisit when user count, corpus, document sizes, workers, image, database, hardware
or quota changes, and after an owner-approved target-runtime observation window.
The existing VERGABE-WP-0019 / RAPPS-WP-0014 human acceptance remains separate.
## Provisional allocation calculation
`config/synthetic-sizing-margins.json` keeps safety factors tunable. After a
complete passing run, derive a candidate envelope without applying it:
```sh
.venv/bin/python scripts/summarize_synthetic_sizing.py /tmp/vergabe-sizing.json \
--margins config/synthetic-sizing-margins.json --output /tmp/vergabe-sizing-envelope.json
```
CPU request is measured aggregate ordinary p95 times 1.5, rounded up to 25m;
CPU limit is observed sample peak times 1.5, rounded up to 100m. Memory request is
peak summed RSS times 1.25 and limit times 1.5, each rounded up to 64Mi. These
are conservative test candidates for this configuration, not measured production
requirements. The normalized per-worker CPU value divides aggregate by configured
workers; it is not a claim that worker load is equal. No database capacity is
folded into the app recommendation. No resource setting is applied by this tool.
The configured cycle creates rather than edits tender/lot/task records. Editing,
startup timing, restart, a distinct burst and target-platform quota/throttling
remain coverage gaps before full pilot sizing acceptance.

View file

@ -0,0 +1,78 @@
"""Derive a provisional local-host envelope, never apply deployment resources."""
import argparse
import json
import math
from pathlib import Path
def percentile(values):
values = sorted(values)
if not values:
raise ValueError("measurement samples missing")
return values[math.ceil(0.95 * len(values)) - 1]
def recommend(result, margin):
samples = result["runtime_samples"]
cpu = []
memory = []
for before, after in zip(samples, samples[1:]):
if before.get("phase") != "ordinary" or after.get("phase") != "ordinary":
continue
if before.get("pids") != after.get("pids"):
continue
elapsed = after["monotonic_seconds"] - before["monotonic_seconds"]
used = after["cpu_seconds"] - before["cpu_seconds"]
if elapsed > 0 and used >= 0:
cpu.append(used / elapsed * 1000)
memory.append(after["rss_bytes"] / 1024**2)
if not cpu or not memory or not result["complete"] or not result["provisional_pass"]:
raise ValueError("incomplete or failed workload cannot establish a sizing envelope")
cpu_p95, cpu_peak, memory_peak = percentile(cpu), max(cpu), max(memory)
def rounded(value, quantum):
return math.ceil(value / quantum) * quantum
return {
"status": "provisional-local-host-only",
"application_observed": {
"cpu_p95_m": cpu_p95,
"cpu_peak_m": cpu_peak,
"rss_sum_peak_mib": memory_peak,
"sample_intervals": len(cpu),
"normalized_cpu_p95_m_per_worker": cpu_p95 / result["config"]["workers"],
},
"suggested_application": {
"cpu_request_m": rounded(cpu_p95 * margin["cpu_request_multiplier"], 25),
"cpu_limit_m": rounded(cpu_peak * margin["cpu_limit_multiplier"], 100),
"memory_request_mib": rounded(memory_peak * margin["memory_request_multiplier"], 64),
"memory_limit_mib": rounded(memory_peak * margin["memory_limit_multiplier"], 64),
},
"margins": margin,
"limits": [
"Per-worker number is normalized aggregate, not an independently sampled worker.",
"RSS sum double-counts shared pages; conservative upper bound, not unique memory.",
"Observed local CPU is hardware/think-time dependent; quota effects untested.",
"Database/container demand is separate and not included in application request.",
"No production apply or human acceptance; validate target runtime before right-sizing.",
],
"revisit": "Changed concurrency, corpus, document size, workers, image, hardware or quota; "
"then repeat under approved target runtime including bursts and restart.",
}
if __name__ == "__main__":
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("result", type=Path)
parser.add_argument("--margins", type=Path, required=True)
parser.add_argument("--output", type=Path, required=True)
args = parser.parse_args()
margins = json.loads(args.margins.read_text())
if any(
not isinstance(v, (int, float)) or not math.isfinite(v) or v < 1 for v in margins.values()
):
raise ValueError("all margins must be finite and at least one")
args.output.write_text(
json.dumps(recommend(json.loads(args.result.read_text()), margins), indent=2) + "\n"
)

479
scripts/synthetic_pilot.py Normal file
View file

@ -0,0 +1,479 @@
#!/usr/bin/env python3
"""Provisional loopback HTTP workload using Gunicorn and disposable PostgreSQL."""
import argparse
import atexit
import concurrent.futures
import hashlib
import http.cookiejar
import json
import math
import os
import platform
import socket
import subprocess
import sys
import tempfile
import threading
import time
import urllib.error
import urllib.parse
import urllib.request
import uuid
from datetime import UTC, datetime
from pathlib import Path
def summarize(rows):
durations = sorted(row["seconds"] for row in rows)
return {
"requests": len(rows),
"errors": sum(not r["ok"] for r in rows),
"p95_seconds": durations[math.ceil(len(durations) * 0.95) - 1] if durations else None,
}
def validate(config):
for key in (
"users",
"iterations",
"tenders",
"document_bytes",
"workers",
"timeout_seconds",
):
if type(config[key]) is not int or config[key] < 1:
raise ValueError(f"{key} must be a positive integer")
for key in (
"think_seconds",
"ordinary_p95_seconds",
"duration_seconds",
"idle_seconds",
):
if not isinstance(config[key], (int, float)) or not math.isfinite(config[key]):
raise ValueError(f"{key} must be finite")
if config["think_seconds"] < 0 or config["ordinary_p95_seconds"] <= 0:
raise ValueError("invalid timing parameter")
if config["duration_seconds"] <= 0 or config["idle_seconds"] < 0:
raise ValueError("duration must be positive and idle nonnegative")
if config["document_bytes"] < 8:
raise ValueError("document_bytes must be at least 8")
return config
def run(config, root):
# Never consume a deployment database, media tree, session or credential.
os.environ["DATABASE_URL"] = os.environ["SYNTHETIC_DATABASE_URL"]
os.environ["SECRET_KEY"] = "synthetic-disposable-not-a-deployment-key"
os.environ["DJANGO_SETTINGS_MODULE"] = "synthetic_settings"
(root / "synthetic_settings.py").write_text(
"from vergabe_teilnahme.settings.base import *\n"
'DEBUG = False\nALLOWED_HOSTS = ["127.0.0.1"]\nNETKINGDOM_ENABLED = False\n'
f"MEDIA_ROOT = {str(root / 'media')!r}\n"
f"ISSUE_FACADE_LOCAL_DB = {str(root / 'issues.db')!r}\n"
'ISSUE_FACADE_GITEA = None\nFORCE_SCRIPT_NAME = None\nMEDIA_URL = "/media/"\n'
'EMAIL_BACKEND = "django.core.mail.backends.locmem.EmailBackend"\n'
"SESSION_COOKIE_SECURE = False\nCSRF_COOKIE_SECURE = False\n"
)
sys.path.insert(0, str(root))
import django
django.setup()
from django.core.management import call_command
from django.db import connections
from vergabe_teilnahme.apps.accounts.models import Mitarbeiter
from vergabe_teilnahme.apps.ausschreibungen.models import Ausschreibung
from vergabe_teilnahme.apps.dokumente.models import Dokument
call_command("migrate", verbosity=0)
for i in range(config["tenders"]):
Ausschreibung.objects.create(titel=f"Synthetic tender {i}", ausschreiber="Synthetic")
users = []
password = uuid.uuid4().hex
for i in range(config["users"]):
user = Mitarbeiter.objects.create_user(username=f"synthetic-{i}", password=password)
users.append(user.username)
with socket.socket() as sock:
sock.bind(("127.0.0.1", 0))
port = sock.getsockname()[1]
base_url = f"http://127.0.0.1:{port}"
env = dict(os.environ, PYTHONPATH=str(root) + os.pathsep + str(Path.cwd()))
server_log = (root / "server.log").open("w")
server = subprocess.Popen(
[
sys.executable,
"-m",
"gunicorn",
"vergabe_teilnahme.wsgi:application",
"--bind",
f"127.0.0.1:{port}",
"--workers",
str(config["workers"]),
"--timeout",
"30",
],
env=env,
stdout=server_log,
stderr=server_log,
)
atexit.register(lambda: server.terminate() if server.poll() is None else None)
for _ in range(100):
try:
with urllib.request.urlopen(base_url + "/health/", timeout=1) as response:
if response.status == 200:
break
except OSError:
if server.poll() is not None:
raise RuntimeError("Gunicorn startup failed; see isolated server log")
time.sleep(0.1)
else:
server.terminate()
raise RuntimeError("Gunicorn readiness timeout")
samples = []
db_samples = []
sampling_done = threading.Event()
phase = {"name": "idle"}
def sample():
ticks_per_second = os.sysconf("SC_CLK_TCK")
while not sampling_done.is_set():
pids = [server.pid]
try:
pids += [
int(x)
for x in Path(f"/proc/{server.pid}/task/{server.pid}/children")
.read_text()
.split()
]
cpu, rss = 0, 0
for pid in pids:
fields = Path(f"/proc/{pid}/stat").read_text().split(") ", 1)[1].split()
cpu += (int(fields[11]) + int(fields[12])) / ticks_per_second
rss += int(fields[21]) * os.sysconf("SC_PAGE_SIZE")
samples.append(
{
"at": datetime.now(UTC).isoformat(),
"monotonic_seconds": time.monotonic(),
"phase": phase["name"],
"cpu_seconds": cpu,
"rss_bytes": rss,
"pids": pids,
}
)
except (OSError, ValueError):
samples.append({"at": datetime.now(UTC).isoformat(), "sample_error": True})
sampling_done.wait(0.5)
def sample_db():
while not sampling_done.is_set():
value = subprocess.check_output(
[
"docker",
"stats",
"--no-stream",
"--format",
"{{json .}}",
os.environ["SYNTHETIC_CONTAINER"],
],
text=True,
)
db_samples.append(
{
"at": datetime.now(UTC).isoformat(),
"phase": phase["name"],
"docker_stats": json.loads(value),
}
)
sampling_done.wait(1)
sampler = threading.Thread(target=sample, daemon=True)
db_sampler = threading.Thread(target=sample_db, daemon=True)
sampler.start()
db_sampler.start()
time.sleep(config["idle_seconds"])
phase["name"] = "ordinary"
deadline = time.monotonic() + config["duration_seconds"]
completed = [0] * config["users"]
barrier = threading.Barrier(config["users"])
def worker(i):
jar = http.cookiejar.CookieJar()
class NoRedirect(urllib.request.HTTPRedirectHandler):
def redirect_request(self, *args, **kwargs):
return None
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(jar), NoRedirect())
rows = []
def request(operation, method, path, data=None, expected=200, transfer=False):
started = time.perf_counter()
try:
headers = {}
body = None
if method == "post":
token = next((c.value for c in jar if c.name == "csrftoken"), "")
fields = dict(data or {}, csrfmiddlewaretoken=token)
if transfer:
boundary = "synthetic-" + uuid.uuid4().hex
body = b""
for name, value in fields.items():
body += (f'--{boundary}\r\nContent-Disposition: form-data; '
f'name="{name}"\r\n\r\n{value}\r\n').encode()
body += (
(f'--{boundary}\r\nContent-Disposition: form-data; '
'name="datei"; filename="synthetic.pdf"\r\n'
'Content-Type: application/pdf\r\n\r\n').encode()
+ b"%PDF-1.4"
+ b"x" * (config["document_bytes"] - 8)
+ f"\r\n--{boundary}--\r\n".encode()
)
headers["Content-Type"] = f"multipart/form-data; boundary={boundary}"
else:
body = urllib.parse.urlencode(fields).encode()
headers["Referer"] = base_url + path
req = urllib.request.Request(
base_url + path, data=body, headers=headers, method=method.upper()
)
try:
response = opener.open(req, timeout=config["timeout_seconds"])
except urllib.error.HTTPError as exc:
response = exc
received = len(response.read())
status = response.status
ok = status == expected
response.close()
except Exception:
status, ok, received = None, False, 0
rows.append(
{
"user": i,
"operation": operation,
"transfer": transfer,
"seconds": time.perf_counter() - started,
"status": status,
"ok": ok,
"response_bytes": received,
}
)
return ok
request("login-form", "get", "/accounts/login/")
barrier.wait(timeout=30)
request(
"login",
"post",
"/accounts/login/",
{"username": users[i], "password": password},
302,
)
for n in range(config["iterations"]):
if time.monotonic() >= deadline:
break
request("list", "get", "/ausschreibungen/")
request("search", "get", "/ausschreibungen/?q=Synthetic")
title = f"Workload-{i}-{n}"
created = request(
"tender-create",
"post",
"/ausschreibungen/neu/",
{"titel": title, "ausschreiber": "Synthetic"},
302,
)
tender = Ausschreibung.objects.filter(titel=title).first() if created else None
if tender:
base = f"/ausschreibungen/{tender.pk}/"
request("detail", "get", base)
request(
"lot-create",
"post",
base + "lose/neu/",
{"losnummer": "1", "lostitel": "Synthetic lot"},
302,
)
request(
"task-create",
"post",
base + "aufgaben/neu/",
{"titel": "Synthetic task", "typ": "fachlich", "prioritaet": 2},
302,
)
request(
"document-upload",
"post",
base + "dokumente/hochladen/",
{"kategorie": "intern", "version": "1.0"},
302,
True,
)
document = Dokument.objects.filter(ausschreibung=tender).first()
if document:
request("document-download", "get", document.datei.url, transfer=True)
completed[i] += 1
time.sleep(config["think_seconds"])
connections.close_all()
return rows
started = datetime.now(UTC).isoformat()
before = time.perf_counter()
with concurrent.futures.ThreadPoolExecutor(max_workers=config["users"]) as pool:
rows = [r for group in pool.map(worker, range(config["users"])) for r in group]
ordinary = summarize([r for r in rows if not r["transfer"]])
transfers = summarize([r for r in rows if r["transfer"]])
complete = all(completed) and len(rows) == sum(completed) * 8 + config["users"] * 2
result = {
"schema_version": 1,
"started_at": started,
"finished_at": datetime.now(UTC).isoformat(),
"duration_seconds": time.perf_counter() - before,
"config": config,
"environment": {
"python": platform.python_version(),
"django": django.get_version(),
"platform": platform.platform(),
"database": "disposable PostgreSQL 16 container",
"transport": "loopback HTTP Gunicorn",
"workers": config["workers"],
},
"ordinary": ordinary,
"document_transfer": transfers,
"complete": complete,
"provisional_pass": complete
and ordinary["errors"] == 0
and transfers["errors"] == 0
and ordinary["p95_seconds"] <= config["ordinary_p95_seconds"],
"limitations": [
"No browser, ingress, TLS, SSO or application container quota",
"No CPU/memory allocation recommendation or human acceptance",
"No timed startup/restart or distinct burst; configured idle and ordinary phases only",
],
"operations": rows,
"runtime_samples": samples,
"database_samples": db_samples,
"completed_iterations": completed,
}
sampling_done.set()
sampler.join()
db_sampler.join()
server.terminate()
server.wait(timeout=10)
server_log.close()
connections.close_all()
return result
if __name__ == "__main__":
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--config", type=Path, required=True)
parser.add_argument("--output", type=Path, required=True)
args = parser.parse_args()
config = validate(json.loads(args.config.read_text()))
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
name = "vergabe-synthetic-" + uuid.uuid4().hex[:12]
# Cached image only, loopback random host port, no existing volume or database.
subprocess.run(
[
"docker",
"run",
"--detach",
"--pull=never",
"--name",
name,
"--publish",
"127.0.0.1::5432",
"--env",
"POSTGRES_DB=synthetic",
"--env",
"POSTGRES_USER=synthetic",
"--env",
"POSTGRES_PASSWORD=synthetic-local-only",
"postgres:16-alpine",
],
check=True,
stdout=subprocess.DEVNULL,
)
try:
for _ in range(100):
ready = subprocess.run(
["docker", "exec", name, "pg_isready", "-U", "synthetic"],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
if ready.returncode == 0:
break
time.sleep(0.1)
else:
raise RuntimeError("isolated PostgreSQL startup failed")
port = (
subprocess.check_output(["docker", "port", name, "5432/tcp"], text=True)
.strip()
.split(":")[-1]
)
os.environ["SYNTHETIC_CONTAINER"] = name
os.environ["SYNTHETIC_DATABASE_URL"] = (
f"postgres://synthetic:synthetic-local-only@127.0.0.1:{port}/synthetic"
)
with tempfile.TemporaryDirectory(prefix="vergabe-synthetic-") as directory:
result = run(config, Path(directory))
result["postgres_image_id"] = subprocess.check_output(
["docker", "inspect", "--format", "{{.Image}}", name], text=True
).strip()
result["postgres_final_stats"] = json.loads(
subprocess.check_output(
["docker", "stats", "--no-stream", "--format", "{{json .}}", name],
text=True,
)
)
finally:
subprocess.run(
["docker", "rm", "--force", "--volumes", name],
check=True,
stdout=subprocess.DEVNULL,
)
result["source_revision"] = subprocess.check_output(
["git", "rev-parse", "HEAD"], text=True
).strip()
result["driver_sha256"] = hashlib.sha256(Path(__file__).read_bytes()).hexdigest()
app_files = sorted(Path("vergabe_teilnahme").rglob("*.py")) + [
Path("uv.lock"),
Path("pyproject.toml"),
]
result["app_source_sha256"] = hashlib.sha256(
b"".join(str(p).encode() + b"\0" + p.read_bytes() for p in app_files)
).hexdigest()
result["host"] = {
"logical_cpus": os.cpu_count(),
"affinity_cpus": len(os.sched_getaffinity(0)),
"cpu_model": next(
(
x.split(":", 1)[1].strip()
for x in Path("/proc/cpuinfo").read_text().splitlines()
if x.startswith("model name")
),
"unknown",
),
"memory_total": next(
x for x in Path("/proc/meminfo").read_text().splitlines() if x.startswith("MemTotal:")
),
"cgroup_cpu_max": Path("/sys/fs/cgroup/cpu.max").read_text().strip()
if Path("/sys/fs/cgroup/cpu.max").exists()
else "unavailable",
"app_cpu_quota": "not imposed by harness",
}
result["config_sha256"] = hashlib.sha256(args.config.read_bytes()).hexdigest()
args.output.write_text(json.dumps(result, indent=2) + "\n")
print(
json.dumps(
{
k: result[k]
for k in (
"ordinary",
"document_transfer",
"complete",
"provisional_pass",
)
}
)
)
raise SystemExit(0 if result["provisional_pass"] else 1)

View file

@ -0,0 +1,98 @@
import importlib.util
import unittest
from pathlib import Path
spec = importlib.util.spec_from_file_location(
"pilot", Path(__file__).with_name("synthetic_pilot.py")
)
pilot = importlib.util.module_from_spec(spec)
spec.loader.exec_module(pilot)
class SyntheticContractTests(unittest.TestCase):
def config(self):
return dict(
users=2,
workers=2,
iterations=20,
tenders=100,
document_bytes=1024,
think_seconds=0.2,
timeout_seconds=30,
ordinary_p95_seconds=2,
duration_seconds=180,
idle_seconds=30,
)
def test_invalid_workload_fails_before_runtime(self):
for key, value in [
("users", 0),
("users", True),
("workers", -1),
("iterations", 1.2),
("document_bytes", 7),
("timeout_seconds", 0),
("duration_seconds", 0),
("idle_seconds", -1),
("ordinary_p95_seconds", float("nan")),
("think_seconds", -1),
]:
with self.subTest(key=key, value=value), self.assertRaises(ValueError):
pilot.validate(self.config() | {key: value})
def test_nearest_rank_p95_and_errors_include_failed_requests(self):
rows = [{"seconds": n, "ok": n != 20} for n in range(1, 21)]
self.assertEqual(pilot.summarize(rows), {"requests": 20, "errors": 1, "p95_seconds": 19})
def test_empty_measurement_is_unknown(self):
self.assertIsNone(pilot.summarize([])["p95_seconds"])
def test_default_is_two_users_and_two_seconds(self):
self.assertEqual(pilot.validate(self.config())["users"], 2)
self.assertEqual(self.config()["ordinary_p95_seconds"], 2)
class SizingTests(unittest.TestCase):
def test_failed_measurement_does_not_recommend(self):
spec = importlib.util.spec_from_file_location(
"sizing", Path(__file__).with_name("summarize_synthetic_sizing.py")
)
sizing = importlib.util.module_from_spec(spec)
spec.loader.exec_module(sizing)
with self.assertRaises(ValueError):
sizing.recommend({"runtime_samples": [], "complete": False}, {})
def test_cpu_delta_and_rounding(self):
spec = importlib.util.spec_from_file_location(
"sizing", Path(__file__).with_name("summarize_synthetic_sizing.py")
)
sizing = importlib.util.module_from_spec(spec)
spec.loader.exec_module(sizing)
rows = [
dict(
phase="ordinary",
pids=[1],
monotonic_seconds=i,
cpu_seconds=i * 0.1,
rss_bytes=100 * 1024**2,
)
for i in range(3)
]
margin = dict(
cpu_request_multiplier=1.5,
cpu_limit_multiplier=1.5,
memory_request_multiplier=1.25,
memory_limit_multiplier=1.5,
)
result = sizing.recommend(
dict(runtime_samples=rows, complete=True, provisional_pass=True, config={"workers": 2}),
margin,
)
self.assertEqual(
result["suggested_application"],
dict(cpu_request_m=150, cpu_limit_m=200, memory_request_mib=128, memory_limit_mib=192),
)
if __name__ == "__main__":
unittest.main()