diff --git a/AGENTS.md b/AGENTS.md index 5c14485..b66db01 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -19,6 +19,7 @@ simulation. No install step. ```bash python3 -m unittest discover -s simulations/gadm02 # must pass before committing model changes cd simulations/gadm02 && python3 run_scenarios.py && python3 sensitivity.py +cd simulations/gadm02 && python3 gsa.py # global sensitivity, ~5 min ``` After changing `simulations/` or `scenarios/`, regenerate the results and diff --git a/models/GADM-0.2.md b/models/GADM-0.2.md index 18915c1..95b6a69 100644 --- a/models/GADM-0.2.md +++ b/models/GADM-0.2.md @@ -147,6 +147,12 @@ Constitutional floors, checked separately: \(G \ge 0.2,\; E \ge 0.2,\; P \le 0.7 ### 3.4 Calibration +**Time unit:** one period = one year (ASSEMBLY-WP-0005-T01), so all rates are +per year and the horizon is 100 years. The implied timescales are mostly +plausible: commons obsolescence about 25 years (`dX`), physical capital about +20 years (`dK`), cohesion adjustment about 5 years (`srate`). Organism +capability, at about 17 years (`dO`), is the main exception (§ 5.1). + There is no empirical calibration. Three coefficients were adjusted so that the **baseline** stays near its initial state, instead of trivially collapsing or saturating: @@ -188,10 +194,13 @@ candidate hypotheses, not claims about economies. 2. **Dividends stabilize demand, not agency.** Dividend abundance raises B from 0.59 to 0.69, but G, P and VFAC barely move. In this model, claims alone do not counter domination or commons erosion. -3. **Reuse works as a bundle.** The commons-rich regime is the strongest - single-lever regime. However, `rho0` alone is not a top driver in the - sensitivity analysis (rank 20 of 80 at baseline). The effect comes from - ρ together with investment I_X, open lineages and standards. +3. **Reuse matters through large shifts, not interaction.** The + commons-rich regime is the strongest single-lever regime, but `rho0` alone + is not a top driver (rank 20 of 80 one-at-a-time). The original reading, + "reuse works as a bundle", was **refuted** by the global analysis (§ 5.1): + the bundle shows almost no interaction. The scenario's effect comes from + moving I_X, ρ_0 and `lin_open` by 100–200 %, far outside local + sensitivity ranges. 4. **Organism governance dominates the generative regime.** Concentrating lineages (`L`, `lin_open`, `m` low) costs about two thirds of the generative regime's VFAC gain over baseline (0.65 → 0.53, baseline 0.46). @@ -235,9 +244,40 @@ Readings: - **More experimentation can hurt** in the generative regime (+20 % `v0` → lower VFAC). This happens through the epistemic brake B3 (link #33, a contested link). -- **One-at-a-time analysis misses interactions.** Finding 3 (reuse as a - bundle) is an interaction effect. A global method (e.g. Sobol, or Morris - screening) is a GADM-0.3 candidate. +- One-at-a-time analysis cannot detect interactions. The global analysis + in § 5.1 checks for them. + +### 5.1 Global sensitivity (ASSEMBLY-WP-0005-T01) + +Full tables: `simulations/gadm02/results/gsa.md`. Method: Morris screening +over all 80 parameters (20 trajectories), then Sobol indices for the top 10 +plus the reuse bundle. Parameters vary ±50 %, N = 256, with bootstrap 95 % +intervals. S1 uses the Saltelli (2010) estimator with centred outputs; ST +uses the Jansen estimator. The uncentred S1 estimators were tried first and +were unusable (VFAC's mean is large relative to its spread). + +| | Baseline | Generative regime | +|---|---|---| +| Top ST | `dO` 0.34, `p7` 0.13, `ell` 0.10, `wL0` 0.07, `IX` 0.07 | `m` 0.27, `L` 0.13, `ereg` 0.13, `p7` 0.07, `expl` 0.06 | +| ΣS1 (analysed set) | 1.04 | 1.06 | +| Reuse bundle ΣS1 / ΣST | 0.09 / 0.11 | 0.08 / 0.09 | + +Readings: + +1. **The model is nearly additive** within ±50 %. ST ≈ S1 for every analysed + parameter, and ΣS1 ≈ 1. Interactions are small, so the one-at-a-time + rankings in § 5 are confirmed. +2. **The "reuse as a bundle" reading is refuted.** The bundle explains about + 10 % of variance, with negligible interaction (ΣST − ΣS1 ≈ 0.01–0.02). + Reading 3 in § 4 has been corrected. +3. **Organism governance remains the dominant lever in the generative + regime.** Membrane control `m` alone explains roughly 27–32 % of variance, + and lineage diffusion `L` a further 13–16 %. Reading 4 in § 4 holds + globally. +4. **Calibration priority: `dO`.** Organism-capability depreciation explains + about a third of baseline variance. At 0.06 per year it implies a 17-year + lifetime, which is plausibly too long for AI systems. It should be + calibrated first. --- @@ -261,4 +301,5 @@ Readings: ```bash python3 -m unittest discover -s simulations/gadm02 cd simulations/gadm02 && python3 run_scenarios.py && python3 sensitivity.py +cd simulations/gadm02 && python3 gsa.py # ~5 min; --quick for a smoke run ``` diff --git a/simulations/README.md b/simulations/README.md index 8258b47..ede06c8 100644 --- a/simulations/README.md +++ b/simulations/README.md @@ -5,7 +5,7 @@ model version it implements. | Directory | Model | Entry points | |-----------|-------|--------------| -| `gadm02/` | GADM-0.2 (`models/GADM-0.2.md`) | `run_scenarios.py`, `sensitivity.py`, `test_gadm02.py` | +| `gadm02/` | GADM-0.2 (`models/GADM-0.2.md`) | `run_scenarios.py`, `sensitivity.py`, `gsa.py`, `test_gadm02.py` | Python 3.10+, standard library only. @@ -13,6 +13,7 @@ Python 3.10+, standard library only. python3 -m unittest discover -s simulations/gadm02 # tests cd simulations/gadm02 && python3 run_scenarios.py # → results/summary.md + CSVs cd simulations/gadm02 && python3 sensitivity.py # → results/sensitivity.md +cd simulations/gadm02 && python3 gsa.py # → results/gsa.md (~5 min; --quick) ``` Results under `results/` are committed so that model documents can cite them. diff --git a/simulations/gadm02/gadm02.py b/simulations/gadm02/gadm02.py index 488d375..944c90d 100644 --- a/simulations/gadm02/gadm02.py +++ b/simulations/gadm02/gadm02.py @@ -12,6 +12,7 @@ from __future__ import annotations import math MODEL_VERSION = "GADM-0.2" +TIME_UNIT = "year" # one period = one year (ASSEMBLY-WP-0005-T01); rates are per year STOCKS = ("H", "X", "Kp", "Ko", "F", "G", "N", "Q", "P", "E", "S", "A") diff --git a/simulations/gadm02/gsa.py b/simulations/gadm02/gsa.py new file mode 100644 index 0000000..1bd3df6 --- /dev/null +++ b/simulations/gadm02/gsa.py @@ -0,0 +1,149 @@ +"""Global sensitivity analysis of final VFAC: Morris screening + Sobol indices. + +Morris (elementary effects, p = 4 levels) screens all parameters. Sobol +first-order (S1, Saltelli 2010 estimator) and total (ST, Jansen estimator) +indices with bootstrap +95 % intervals, are then computed for the top Morris parameters plus the +reuse bundle (rho0, IX, lin_open, std), so the "reuse works as a bundle" +reading in models/GADM-0.2.md can be tested directly. + +Ranges: each parameter varies ±50 % around its scenario value; levers are +clipped to [0, 1]; zero-valued parameters vary in [0, 0.1]. + +Usage: python3 simulations/gadm02/gsa.py [--quick] +""" + +from __future__ import annotations + +import random +import statistics +import sys +from pathlib import Path + +import gadm02 as g +from scenarios import load_all + +OUT = Path(__file__).resolve().parent / "results" +LEVERS = {"m", "L", "lin_open", "std", "disc", "expl", "x_att", "auto", "dshare", "cp", "od"} +BUNDLE = ["rho0", "IX", "lin_open", "std"] +SEED = 20261012 + + +def ranges(base: dict) -> dict[str, tuple[float, float]]: + out = {} + for k, v in base.items(): + lo, hi = (0.0, 0.1) if v == 0 else (v * 0.5, v * 1.5) + if k in LEVERS: + lo, hi = max(0.0, lo), min(1.0, hi) + out[k] = (lo, hi) + return out + + +def model(base: dict, rng: dict, names: list[str], u: list[float], init: dict) -> float: + p = dict(base) + for k, x in zip(names, u): + lo, hi = rng[k] + p[k] = lo + x * (hi - lo) + return g.simulate(p, init)[-1]["VFAC"] + + +def morris(base, init, rng, r, rnd): + names = list(base) + k, delta = len(names), 2 / 3 + effects = {n: [] for n in names} + for _ in range(r): + x = [rnd.choice((0.0, 1 / 3)) for _ in range(k)] + y = model(base, rng, names, x, init) + for i in rnd.sample(range(k), k): + x[i] += delta + y2 = model(base, rng, names, x, init) + effects[names[i]].append((y2 - y) / delta) + y = y2 + res = [] + for n, ee in effects.items(): + res.append((n, statistics.fmean(abs(e) for e in ee), statistics.fmean(ee), statistics.pstdev(ee))) + res.sort(key=lambda t: -t[1]) + return res + + +def sobol(base, init, rng, names, n, rnd, boot=200): + k = len(names) + A = [[rnd.random() for _ in range(k)] for _ in range(n)] + B = [[rnd.random() for _ in range(k)] for _ in range(n)] + fA = [model(base, rng, names, a, init) for a in A] + fB = [model(base, rng, names, b, init) for b in B] + fAB = [] + for i in range(k): + fAB.append([model(base, rng, names, A[j][:i] + [B[j][i]] + A[j][i + 1:], init) for j in range(n)]) + + def indices(idx): + a = [fA[j] for j in idx] + b = [fB[j] for j in idx] + var = statistics.pvariance(a + b) or 1e-12 + mu = statistics.fmean(a + b) + out = [] + for i in range(k): + ab = [fAB[i][j] for j in idx] + # Saltelli 2010 with centred outputs: VFAC has a large mean relative to its + # spread, and centring removes that mean from the product's variance. + s1 = sum((bb - mu) * (x - aa) for aa, bb, x in zip(a, b, ab)) / len(idx) / var + st = sum((aa - x) ** 2 for aa, x in zip(a, ab)) / (2 * len(idx)) / var + out.append((s1, st)) + return out + + point = indices(range(n)) + reps = [indices([rnd.randrange(n) for _ in range(n)]) for _ in range(boot)] + + def ci(i, which): + vals = sorted(r[i][which] for r in reps) + return vals[int(0.025 * boot)], vals[int(0.975 * boot) - 1] + + return [(names[i], point[i][0], ci(i, 0), point[i][1], ci(i, 1)) for i in range(k)] + + +def main() -> None: + quick = "--quick" in sys.argv + r, n, top = (4, 32, 6) if quick else (20, 256, 10) + OUT.mkdir(exist_ok=True) + scs = load_all() + lines = [f"# {g.MODEL_VERSION} global sensitivity analysis", "", + "Generated by `simulations/gadm02/gsa.py`" + (" (--quick)" if quick else "") + ".", + f"Output: final VFAC (t = 100 {g.TIME_UNIT}s). Parameters vary ±50 % around the scenario", + "value (levers clipped to [0, 1]; zero-valued parameters in [0, 0.1]).", + f"Morris: {r} trajectories, 4 levels. Sobol: S1 Saltelli (2010, centred), ST Jansen, N = {n},", + f"bootstrap 95 % intervals, seed {SEED}. Parameters outside the Sobol set stay at", + "their scenario values.", "", + "μ* ranks importance. A σ that is large relative to μ* indicates", + "interactions or non-linearity. ST − S1 is the share of variance due to", + "interactions involving that parameter.", ""] + for name in ("baseline", "generative_regime"): + rnd = random.Random(SEED) + sc = scs[name] + base = {**g.PARAMS, **sc["params"]} + rng = ranges(base) + mres = morris(base, sc["init"], rng, r, rnd) + lines += [f"## {sc['title']}", "", "### Morris screening (top 15)", "", + "| Rank | Parameter | μ* | μ | σ |", "|---|---|---|---|---|"] + lines += [f"| {i + 1} | `{p}` | {ms:.4f} | {mu:+.4f} | {sd:.4f} |" + for i, (p, ms, mu, sd) in enumerate(mres[:15])] + mrank = {p: i + 1 for i, (p, *_rest) in enumerate(mres)} + lines += ["", "Reuse bundle Morris ranks: " + ", ".join(f"`{b}` {mrank[b]}" for b in BUNDLE), ""] + names = [p for p, *_rest in mres[:top]] + [b for b in BUNDLE if b not in [p for p, *_r in mres[:top]]] + sres = sobol(base, sc["init"], rng, names, n, rnd) + lines += ["### Sobol indices", "", + "| Parameter | S1 | S1 95 % CI | ST | ST 95 % CI | ST − S1 |", "|---|---|---|---|---|---|"] + for p, s1, c1, st, ct in sorted(sres, key=lambda t: -t[3]): + lines.append(f"| `{p}` | {s1:.3f} | [{c1[0]:.3f}, {c1[1]:.3f}] | {st:.3f} | " + f"[{ct[0]:.3f}, {ct[1]:.3f}] | {st - s1:+.3f} |") + b1 = sum(s1 for p, s1, _c, _st, _ct in sres if p in BUNDLE) + bt = sum(st for p, _s1, _c, st, _ct in sres if p in BUNDLE) + tot1 = sum(s1 for _p, s1, *_r in sres) + lines += ["", f"Sum of S1 over analysed parameters: {tot1:.3f}. Reuse bundle: " + f"ΣS1 = {b1:.3f}, ΣST = {bt:.3f}.", ""] + print(f"done {name}", file=sys.stderr) + (OUT / ("gsa-quick.md" if quick else "gsa.md")).write_text("\n".join(lines)) + print("\n".join(lines)) + + +if __name__ == "__main__": + main() diff --git a/simulations/gadm02/results/gsa.md b/simulations/gadm02/results/gsa.md new file mode 100644 index 0000000..8e41063 --- /dev/null +++ b/simulations/gadm02/results/gsa.md @@ -0,0 +1,101 @@ +# GADM-0.2 global sensitivity analysis + +Generated by `simulations/gadm02/gsa.py`. +Output: final VFAC (t = 100 years). Parameters vary ±50 % around the scenario +value (levers clipped to [0, 1]; zero-valued parameters in [0, 0.1]). +Morris: 20 trajectories, 4 levels. Sobol: S1 Saltelli (2010, centred), ST Jansen, N = 256, +bootstrap 95 % intervals, seed 20261012. Parameters outside the Sobol set stay at +their scenario values. + +μ* ranks importance. A σ that is large relative to μ* indicates +interactions or non-linearity. ST − S1 is the share of variance due to +interactions involving that parameter. + +## Baseline (current trends) + +### Morris screening (top 15) + +| Rank | Parameter | μ* | μ | σ | +|---|---|---|---|---| +| 1 | `dO` | 0.0846 | +0.0846 | 0.0229 | +| 2 | `p7` | 0.0485 | +0.0485 | 0.0148 | +| 3 | `ell` | 0.0476 | -0.0476 | 0.0106 | +| 4 | `IX` | 0.0402 | +0.0402 | 0.0159 | +| 5 | `wL0` | 0.0373 | +0.0373 | 0.0112 | +| 6 | `p2` | 0.0353 | -0.0353 | 0.0143 | +| 7 | `n1` | 0.0340 | +0.0340 | 0.0095 | +| 8 | `p5` | 0.0308 | +0.0308 | 0.0106 | +| 9 | `disc` | 0.0291 | -0.0291 | 0.0091 | +| 10 | `g1` | 0.0269 | +0.0269 | 0.0075 | +| 11 | `dX` | 0.0266 | -0.0266 | 0.0053 | +| 12 | `aO` | 0.0266 | -0.0266 | 0.0112 | +| 13 | `p4` | 0.0248 | -0.0248 | 0.0067 | +| 14 | `arec` | 0.0242 | +0.0242 | 0.0131 | +| 15 | `aX` | 0.0241 | +0.0241 | 0.0100 | + +Reuse bundle Morris ranks: `rho0` 24, `IX` 4, `lin_open` 38, `std` 35 + +### Sobol indices + +| Parameter | S1 | S1 95 % CI | ST | ST 95 % CI | ST − S1 | +|---|---|---|---|---|---| +| `dO` | 0.339 | [0.258, 0.426] | 0.337 | [0.289, 0.400] | -0.001 | +| `p7` | 0.146 | [0.099, 0.209] | 0.127 | [0.107, 0.154] | -0.020 | +| `ell` | 0.121 | [0.066, 0.176] | 0.100 | [0.082, 0.122] | -0.022 | +| `wL0` | 0.095 | [0.048, 0.136] | 0.071 | [0.060, 0.085] | -0.024 | +| `IX` | 0.049 | [0.011, 0.094] | 0.071 | [0.057, 0.087] | +0.021 | +| `n1` | 0.049 | [0.015, 0.083] | 0.050 | [0.042, 0.060] | +0.001 | +| `disc` | 0.068 | [0.041, 0.109] | 0.047 | [0.038, 0.058] | -0.021 | +| `p5` | 0.036 | [0.008, 0.071] | 0.046 | [0.038, 0.056] | +0.010 | +| `p2` | 0.044 | [0.004, 0.077] | 0.042 | [0.032, 0.054] | -0.002 | +| `g1` | 0.047 | [0.018, 0.076] | 0.033 | [0.029, 0.039] | -0.013 | +| `rho0` | 0.031 | [0.004, 0.062] | 0.025 | [0.020, 0.030] | -0.007 | +| `std` | 0.010 | [-0.009, 0.028] | 0.012 | [0.010, 0.015] | +0.002 | +| `lin_open` | 0.003 | [-0.007, 0.015] | 0.004 | [0.004, 0.005] | +0.002 | + +Sum of S1 over analysed parameters: 1.039. Reuse bundle: ΣS1 = 0.094, ΣST = 0.112. + +## Generative regime + +### Morris screening (top 15) + +| Rank | Parameter | μ* | μ | σ | +|---|---|---|---|---| +| 1 | `m` | 0.0565 | +0.0565 | 0.0128 | +| 2 | `L` | 0.0422 | +0.0422 | 0.0114 | +| 3 | `ereg` | 0.0405 | +0.0405 | 0.0109 | +| 4 | `expl` | 0.0297 | +0.0297 | 0.0054 | +| 5 | `p7` | 0.0294 | +0.0294 | 0.0127 | +| 6 | `dO` | 0.0276 | +0.0276 | 0.0160 | +| 7 | `p5` | 0.0238 | +0.0238 | 0.0133 | +| 8 | `aX` | 0.0231 | +0.0231 | 0.0080 | +| 9 | `p4` | 0.0215 | -0.0215 | 0.0125 | +| 10 | `n1` | 0.0214 | +0.0214 | 0.0086 | +| 11 | `IX` | 0.0213 | +0.0213 | 0.0144 | +| 12 | `thr` | 0.0208 | -0.0208 | 0.0070 | +| 13 | `arec` | 0.0205 | +0.0205 | 0.0133 | +| 14 | `q0` | 0.0205 | +0.0205 | 0.0065 | +| 15 | `dshare` | 0.0202 | +0.0202 | 0.0063 | + +Reuse bundle Morris ranks: `rho0` 25, `IX` 11, `lin_open` 20, `std` 41 + +### Sobol indices + +| Parameter | S1 | S1 95 % CI | ST | ST 95 % CI | ST − S1 | +|---|---|---|---|---|---| +| `m` | 0.318 | [0.224, 0.399] | 0.268 | [0.217, 0.330] | -0.049 | +| `L` | 0.157 | [0.098, 0.218] | 0.132 | [0.110, 0.164] | -0.025 | +| `ereg` | 0.164 | [0.107, 0.225] | 0.129 | [0.103, 0.156] | -0.035 | +| `p7` | 0.097 | [0.053, 0.143] | 0.069 | [0.058, 0.080] | -0.029 | +| `expl` | 0.086 | [0.050, 0.135] | 0.059 | [0.047, 0.070] | -0.027 | +| `dO` | 0.021 | [-0.019, 0.060] | 0.047 | [0.038, 0.057] | +0.026 | +| `aX` | 0.054 | [0.023, 0.088] | 0.042 | [0.035, 0.051] | -0.012 | +| `p5` | 0.009 | [-0.019, 0.045] | 0.039 | [0.033, 0.049] | +0.031 | +| `lin_open` | 0.055 | [0.024, 0.084] | 0.030 | [0.025, 0.038] | -0.024 | +| `n1` | 0.053 | [0.027, 0.084] | 0.029 | [0.024, 0.034] | -0.024 | +| `p4` | 0.017 | [-0.015, 0.043] | 0.026 | [0.020, 0.033] | +0.009 | +| `rho0` | 0.010 | [-0.017, 0.037] | 0.021 | [0.017, 0.025] | +0.011 | +| `std` | 0.012 | [-0.014, 0.036] | 0.018 | [0.014, 0.023] | +0.006 | +| `IX` | 0.007 | [-0.018, 0.033] | 0.018 | [0.013, 0.022] | +0.010 | + +Sum of S1 over analysed parameters: 1.062. Reuse bundle: ΣS1 = 0.084, ΣST = 0.087. diff --git a/workplans/ASSEMBLY-WP-0005-gadm-0.3.md b/workplans/ASSEMBLY-WP-0005-gadm-0.3.md index 2b3729a..088c133 100644 --- a/workplans/ASSEMBLY-WP-0005-gadm-0.3.md +++ b/workplans/ASSEMBLY-WP-0005-gadm-0.3.md @@ -4,7 +4,7 @@ type: workplan title: "GADM-0.3: global sensitivity, power channels, cohorts" domain: communication repo: assembly-economics -status: proposed +status: active flavor: residual origin: residual origin_ref: ASSEMBLY-WP-0004 @@ -26,7 +26,7 @@ to `ready`. ```task id: ASSEMBLY-WP-0005-T01 -status: todo +status: done priority: high state_hub_task_id: "66f7e439-b903-59e0-897a-446e157bf1a1" ```