ORGREF-WP-0002-T02: first anonymous-consumer inventory

40 cluster workloads pull forge images without imagePullSecrets, 6 ArgoCD
applications read forge Git; local clones are SSH. Live credential checks and
per-entry review remain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Assistant: claude-code
Assistant-Model: opus
Assistant-Process: 5457@bnt-lap001
Assistant-Session: 792bf84a-3783-4f5d-a811-cded61dd2012
This commit is contained in:
tegwick 2026-10-08 16:37:33 +02:00
parent 64486863ab
commit f91cbca461
5 changed files with 8421 additions and 1 deletions

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,46 @@
# Anonymous forge consumers — ORGREF-WP-0002-T02
Machine-readable result: `access-consumers-2026-10-08.json`. Reproduce:
```bash
python3 tools/access_consumers.py --root /home/worsch \
--runtime inventory/runtime-2026-09-28.json \
--output inventory/access-consumers-2026-10-08.json
```
Read-only: 161 local working trees scanned, plus the 2026-09-28 runtime capture
(not a live read). Secrets, ArgoCD repository credentials and node registry
configuration were not inspected.
## What breaks when repositories and packages go private
| Class | Count | Reads | Will need |
| --- | --- | --- | --- |
| cluster-image-pull | 40 workloads in 21 namespaces | container images from the Forgejo registry, **none with `imagePullSecrets`** | pull credentials per namespace, or node-level registry auth (k3s `registries.yaml`), which is unverified |
| cluster-gitops | 6 ArgoCD applications | Git sources on the forge | ArgoCD repository credentials (current state unverified) |
| ci-runner | 27 hits in workflow files | checkouts, reusable actions, registry | runner or job tokens for cross-repo reads |
| api-client | 25 hits | `/api/v1` calls (state-hub, railiance-*, ops-warden, issue-core, …) | per-service tokens; check which calls are anonymous today |
| raw-file-fetch | 68 hits, 63 in `reuse-surface` | raw files by URL | authenticated fetch, or move the published content to a public location |
| raw-file-fetch (campaign) | pr-hall-of-helix, fluid-telegram | Telegram fetching post images | a deliberate public asset location (ORGREF-WP-0002-T01 point 4) |
| package-consumer | 2 (kaizen-agentic, whynot-design) | modules or packages from the forge | package-registry credentials |
Lower risk, review only: `repo-gitops` (15) and `repo-image-reference` (91) are
the same cluster consumers seen from their source repos. `code-reference` (24),
`config-reference` (98) and `build-tooling` (23) are mostly URLs and pushes that
need a per-entry look. `doc-link` (119) means links break for the public, not
consumers. `record-or-test` (103) is evidence and fixtures.
## Not anonymous
All 160 local clones use SSH with the founder's key (`forgejo-remote`). They keep
working as long as that account can read the repositories.
## Still open for T02
- Live verification: k3s node registry auth, ArgoCD repository credentials, and
a fresh runtime capture (the 2026-09-28 capture is the basis here).
- Per-entry review of `api-client`, `code-reference` and `ci-runner` hits, to
tell anonymous reads from authenticated writes.
- Consumers outside these working trees (external sites and links, other
people's clones) cannot be found by scanning; T08's anonymous probe after each
wave is the backstop.

View file

@ -0,0 +1,43 @@
import sys
import unittest
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools"))
from access_consumers import classify, runtime_consumers, target_repo # noqa: E402
class ClassifyTest(unittest.TestCase):
def test_kinds(self):
cases = [
("posts/x.yaml", "/coulomb/pr-hall-of-helix/raw/branch/main/a.png", "url: x", "raw-fetch"),
("src/client.py", "/api/v1/repos/coulomb/x", "u = ...", "api-client"),
(".forgejo/workflows/image.yaml", "/coulomb/x", "uses: x", "ci-action"),
("go.mod", "/coulomb/x", "require x", "package-consumer"),
("k8s/app.yaml", "/coulomb/x.git", " repoURL: https://...", "gitops-source"),
("chart/values.yaml", "/coulomb/x", " repository: forgejo...", "image-pull"),
("README.md", "/coulomb/x", "see", "doc-link"),
("docs/evidence/run.json", "/coulomb/x", "x", "record"),
("Makefile", "/coulomb/x", "push", "build-tooling"),
]
for f, p, line, want in cases:
self.assertEqual(classify(f, p, line), want, f)
def test_target_repo(self):
self.assertEqual(target_repo("/coulomb/key-cape.git"), "coulomb/key-cape")
self.assertEqual(target_repo("/api/v1/repos/coulomb/state-hub/issues"), "coulomb/state-hub")
self.assertEqual(target_repo("/v2/coulomb/flex-auth/manifests/x"), "coulomb/flex-auth")
self.assertEqual(target_repo("/coulomb/activity-core@sha256:ab"), "coulomb/activity-core")
self.assertIsNone(target_repo("/user/login"))
def test_runtime_anonymous_pull(self):
rt = {"applications": [], "workloads": [{
"namespace": "ns", "kind": "Deployment", "name": "api", "image_pull_secret_names": [],
"images": ["forgejo.coulomb.social/coulomb/x@sha256:00"]}]}
(c,) = runtime_consumers(rt)
self.assertEqual(c["class"], "cluster-image-pull")
self.assertTrue(c["credential"].startswith("anonymous"))
self.assertEqual(c["target"], "coulomb/x")
if __name__ == "__main__":
unittest.main()

207
tools/access_consumers.py Normal file
View file

@ -0,0 +1,207 @@
#!/usr/bin/env python3
"""Inventory everything that reads the forge without credentials (ORGREF-WP-0002-T02).
Read-only. Combines the runtime capture (ArgoCD sources and workload images)
with a fresh scan of local working trees for forge URLs, classifies each hit
by how it reads the forge, and groups the result into consumers: the repo or
workload that reads, what it reads, and how. A consumer that reads over SSH
with a personal key is recorded but is not anonymous.
python3 tools/access_consumers.py --root /home/worsch \
--runtime inventory/runtime-2026-09-28.json \
--output inventory/access-consumers-2026-10-08.json
"""
import argparse
import collections
import csv
import json
import os
import re
import subprocess
from pathlib import Path
FORGE_HOSTS = ("forgejo.coulomb.social", "gitea.coulomb.social")
URL = re.compile(
r"(?:https?://)?(?P<host>forgejo\.coulomb\.social|gitea\.coulomb\.social)"
r"(?P<path>/[A-Za-z0-9._~%/@:+-]*)?")
SKIP_DIRS = {".git", "node_modules", "vendor", "history", "archived", "__pycache__",
".venv", "venv", "dist", "build", ".terraform", "rescued"}
SKIP_FILES = {"WORK-RECORDS.md", ".custodian-brief.md"}
MAX_BYTES = 1_000_000
# How a reference reads the forge. First match wins; order matters.
KINDS = [
("record", lambda f, p, line: f.startswith(("docs/evidence/", "evidence/", "tests/", "test/"))
or "/evidence/" in f or "/tests/" in f or f.endswith(("PKG-INFO", ".sum"))),
("raw-fetch", lambda f, p, line: "/raw/" in p or "/media/" in p),
("api-client", lambda f, p, line: p.startswith("/api/")),
("ci-action", lambda f, p, line: ".forgejo/workflows" in f or ".gitea/workflows" in f
or ".github/workflows" in f),
("package-consumer", lambda f, p, line: Path(f).name in {
"go.mod", "go.sum", "pyproject.toml", "requirements.txt", "package.json",
"uv.lock", "Cargo.toml", "setup.cfg"} or "/api/packages/" in p),
("gitops-source", lambda f, p, line: "repoURL" in line or p.endswith(".git")
and f.endswith((".yaml", ".yml"))),
("image-pull", lambda f, p, line: bool(re.search(r"(image|repository|FROM)\b", line))
or "@sha256:" in line or Path(f).name.startswith(("Dockerfile", "Containerfile"))),
("doc-link", lambda f, p, line: f.endswith((".md", ".rst", ".txt", ".html"))),
("build-tooling", lambda f, p, line: Path(f).name in {"Makefile", "justfile"}
or f.endswith((".sh", ".mk"))),
("code-reference", lambda f, p, line: f.endswith((".py", ".go", ".ts", ".js", ".rs"))),
("config-reference", lambda f, p, line: f.endswith((".yaml", ".yml", ".json", ".toml",
".ini", ".service", ".template", ".example"))),
]
def classify(file, path, line):
for kind, test in KINDS:
if test(file, path or "", line):
return kind
return "other"
def target_repo(path):
"""org/repo named by a forge URL path, if any."""
parts = [p for p in (path or "").split("/") if p]
if parts[:2] == ["api", "v1"] and len(parts) >= 5 and parts[2] == "repos":
parts = parts[3:]
elif parts and parts[0] in {"api", "v2"}:
parts = parts[1:]
if len(parts) >= 2 and parts[0] not in {"user", "explore", "assets"}:
return f"{parts[0]}/{parts[1].removesuffix('.git').split('@')[0].split(':')[0]}"
return None
def scan_repo(repo):
hits = []
for dirpath, dirnames, filenames in os.walk(repo):
dirnames[:] = [d for d in dirnames if d not in SKIP_DIRS and not d.startswith(".")
or d in {".forgejo", ".gitea", ".github"}]
for name in filenames:
if name in SKIP_FILES:
continue
full = Path(dirpath) / name
try:
if full.is_symlink() or full.stat().st_size > MAX_BYTES:
continue
text = full.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError):
continue
rel = str(full.relative_to(repo))
for number, line in enumerate(text.splitlines(), 1):
for m in URL.finditer(line):
hits.append({
"file": rel, "line": number, "host": m.group("host"),
"kind": classify(rel, m.group("path") or "", line),
"target": target_repo(m.group("path")),
})
return hits
def remote_form(repo):
try:
url = subprocess.run(["git", "-C", str(repo), "remote", "get-url", "origin"],
capture_output=True, text=True, check=True).stdout.strip()
except (subprocess.CalledProcessError, OSError):
return None
if url.startswith(("forgejo-remote:", "ssh://", "git@")):
return "ssh-key"
if any(h in url for h in FORGE_HOSTS):
return "https"
return "other"
def runtime_consumers(runtime):
out = []
for app in runtime.get("applications", []):
for src in app.get("sources", []):
u = src.get("repoURL", "")
m = URL.search(u)
if m:
out.append({"consumer": f"argocd:{app['name']}", "class": "cluster-gitops",
"kind": "gitops-source", "target": target_repo(m.group("path")),
"credential": "unknown (ArgoCD repo credentials not inspected)"})
for w in runtime.get("workloads", []):
for image in w.get("images", []):
m = URL.search(image)
if m:
secrets = w.get("image_pull_secret_names") or []
out.append({"consumer": f"k3s:{w['namespace']}/{w['kind']}/{w['name']}",
"class": "cluster-image-pull", "kind": "image-pull",
"target": target_repo(m.group("path")),
"credential": "pull secret named: " + ",".join(secrets)
if secrets else "anonymous (no imagePullSecrets)"})
return out
CLASS_OF_KIND = {
"raw-fetch": "raw-file-fetch", "api-client": "api-client", "ci-action": "ci-runner",
"package-consumer": "package-consumer", "gitops-source": "repo-gitops",
"image-pull": "repo-image-reference", "doc-link": "doc-link", "other": "other",
"record": "record-or-test", "build-tooling": "build-tooling",
"code-reference": "code-reference", "config-reference": "config-reference",
}
def main():
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument("--root", default="/home/worsch")
ap.add_argument("--runtime", required=True)
ap.add_argument("--assignments", default="taxonomy/assignments.csv")
ap.add_argument("--output", required=True)
args = ap.parse_args()
stewards = {}
with open(args.assignments, newline="") as fh:
for row in csv.DictReader(fh):
stewards[row["repository"]] = row["proposed_org"]
runtime = json.load(open(args.runtime))
consumers = runtime_consumers(runtime)
remotes = collections.Counter()
scanned = []
for repo in sorted(Path(args.root).iterdir()):
if not (repo / ".git").exists() or repo.name.startswith("rescued-"):
continue
scanned.append(repo.name)
remotes[remote_form(repo) or "none"] += 1
grouped = collections.defaultdict(list)
for h in scan_repo(repo):
grouped[(h["kind"], h["target"])].append(f"{h['file']}:{h['line']}")
for (kind, target), where in sorted(grouped.items(), key=lambda kv: (kv[0][0], str(kv[0][1]))):
consumers.append({"consumer": f"repo:{repo.name}", "class": CLASS_OF_KIND[kind],
"kind": kind, "target": target, "locations": where[:10],
"location_count": len(where)})
for c in consumers:
name = (c.get("target") or "").split("/")[-1]
c["target_steward"] = stewards.get(name, "unassigned")
summary = collections.Counter(c["class"] for c in consumers)
report = {
"schema_version": "0.1",
"workplan": "ORGREF-WP-0002-T02",
"runtime_capture": os.path.basename(args.runtime),
"scanned_repositories": len(scanned),
"local_remote_forms": dict(remotes),
"summary_by_class": dict(sorted(summary.items())),
"anonymous_image_pulls": sum(1 for c in consumers if c["class"] == "cluster-image-pull"
and c["credential"].startswith("anonymous")),
"consumers": consumers,
"limits": [
"Runtime classes come from the runtime capture named above, not a live read.",
"ArgoCD repository credentials and pull-secret contents were not inspected; "
"a named pull secret is not proof that it authenticates to the forge.",
"Only literal forge hostnames are found; computed URLs and consumers outside "
"these working trees (external sites, Telegram, other people's clones) are not.",
"history/ and archived/ directories are skipped; they record, they do not read.",
],
}
Path(args.output).write_text(json.dumps(report, indent=1, sort_keys=False) + "\n")
print(json.dumps({k: report[k] for k in ("scanned_repositories", "local_remote_forms",
"summary_by_class", "anonymous_image_pulls")},
indent=1))
if __name__ == "__main__":
main()

View file

@ -122,7 +122,7 @@ Record a founder decision covering:
```task
id: ORGREF-WP-0002-T02
status: todo
status: progress
priority: high
state_hub_task_id: "ace26a22-50e2-58c6-b4ad-85d95c284fce"
```
@ -146,6 +146,13 @@ Output a machine-readable `inventory/access-consumers-<date>.json` plus a
summary listing, per consumer class, the owner and the proposed credential
kind. Uncertain entries are flagged, not dropped.
2026-10-08: first pass done. `tools/access_consumers.py` produced
`inventory/access-consumers-2026-10-08.json`, summarised in
`inventory/access-consumers.md`. Headline: 40 cluster workloads pull forge images
with no `imagePullSecrets`; 6 ArgoCD applications read forge Git; local clones
are SSH and unaffected. Open: live check of node registry auth and ArgoCD
credentials, and a per-entry review of api-client, code and CI hits.
```task
id: ORGREF-WP-0002-T03
status: todo