ORGREF-WP-0002-T02: first anonymous-consumer inventory
40 cluster workloads pull forge images without imagePullSecrets, 6 ArgoCD applications read forge Git; local clones are SSH. Live credential checks and per-entry review remain. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 5457@bnt-lap001 Assistant-Session: 792bf84a-3783-4f5d-a811-cded61dd2012
This commit is contained in:
parent
64486863ab
commit
f91cbca461
5 changed files with 8421 additions and 1 deletions
8117
inventory/access-consumers-2026-10-08.json
Normal file
8117
inventory/access-consumers-2026-10-08.json
Normal file
File diff suppressed because it is too large
Load diff
46
inventory/access-consumers.md
Normal file
46
inventory/access-consumers.md
Normal file
|
|
@ -0,0 +1,46 @@
|
|||
# Anonymous forge consumers — ORGREF-WP-0002-T02
|
||||
|
||||
Machine-readable result: `access-consumers-2026-10-08.json`. Reproduce:
|
||||
|
||||
```bash
|
||||
python3 tools/access_consumers.py --root /home/worsch \
|
||||
--runtime inventory/runtime-2026-09-28.json \
|
||||
--output inventory/access-consumers-2026-10-08.json
|
||||
```
|
||||
|
||||
Read-only: 161 local working trees scanned, plus the 2026-09-28 runtime capture
|
||||
(not a live read). Secrets, ArgoCD repository credentials and node registry
|
||||
configuration were not inspected.
|
||||
|
||||
## What breaks when repositories and packages go private
|
||||
|
||||
| Class | Count | Reads | Will need |
|
||||
| --- | --- | --- | --- |
|
||||
| cluster-image-pull | 40 workloads in 21 namespaces | container images from the Forgejo registry, **none with `imagePullSecrets`** | pull credentials per namespace, or node-level registry auth (k3s `registries.yaml`), which is unverified |
|
||||
| cluster-gitops | 6 ArgoCD applications | Git sources on the forge | ArgoCD repository credentials (current state unverified) |
|
||||
| ci-runner | 27 hits in workflow files | checkouts, reusable actions, registry | runner or job tokens for cross-repo reads |
|
||||
| api-client | 25 hits | `/api/v1` calls (state-hub, railiance-*, ops-warden, issue-core, …) | per-service tokens; check which calls are anonymous today |
|
||||
| raw-file-fetch | 68 hits, 63 in `reuse-surface` | raw files by URL | authenticated fetch, or move the published content to a public location |
|
||||
| raw-file-fetch (campaign) | pr-hall-of-helix, fluid-telegram | Telegram fetching post images | a deliberate public asset location (ORGREF-WP-0002-T01 point 4) |
|
||||
| package-consumer | 2 (kaizen-agentic, whynot-design) | modules or packages from the forge | package-registry credentials |
|
||||
|
||||
Lower risk, review only: `repo-gitops` (15) and `repo-image-reference` (91) are
|
||||
the same cluster consumers seen from their source repos. `code-reference` (24),
|
||||
`config-reference` (98) and `build-tooling` (23) are mostly URLs and pushes that
|
||||
need a per-entry look. `doc-link` (119) means links break for the public, not
|
||||
consumers. `record-or-test` (103) is evidence and fixtures.
|
||||
|
||||
## Not anonymous
|
||||
|
||||
All 160 local clones use SSH with the founder's key (`forgejo-remote`). They keep
|
||||
working as long as that account can read the repositories.
|
||||
|
||||
## Still open for T02
|
||||
|
||||
- Live verification: k3s node registry auth, ArgoCD repository credentials, and
|
||||
a fresh runtime capture (the 2026-09-28 capture is the basis here).
|
||||
- Per-entry review of `api-client`, `code-reference` and `ci-runner` hits, to
|
||||
tell anonymous reads from authenticated writes.
|
||||
- Consumers outside these working trees (external sites and links, other
|
||||
people's clones) cannot be found by scanning; T08's anonymous probe after each
|
||||
wave is the backstop.
|
||||
43
tests/test_access_consumers.py
Normal file
43
tests/test_access_consumers.py
Normal file
|
|
@ -0,0 +1,43 @@
|
|||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools"))
|
||||
from access_consumers import classify, runtime_consumers, target_repo # noqa: E402
|
||||
|
||||
|
||||
class ClassifyTest(unittest.TestCase):
|
||||
def test_kinds(self):
|
||||
cases = [
|
||||
("posts/x.yaml", "/coulomb/pr-hall-of-helix/raw/branch/main/a.png", "url: x", "raw-fetch"),
|
||||
("src/client.py", "/api/v1/repos/coulomb/x", "u = ...", "api-client"),
|
||||
(".forgejo/workflows/image.yaml", "/coulomb/x", "uses: x", "ci-action"),
|
||||
("go.mod", "/coulomb/x", "require x", "package-consumer"),
|
||||
("k8s/app.yaml", "/coulomb/x.git", " repoURL: https://...", "gitops-source"),
|
||||
("chart/values.yaml", "/coulomb/x", " repository: forgejo...", "image-pull"),
|
||||
("README.md", "/coulomb/x", "see", "doc-link"),
|
||||
("docs/evidence/run.json", "/coulomb/x", "x", "record"),
|
||||
("Makefile", "/coulomb/x", "push", "build-tooling"),
|
||||
]
|
||||
for f, p, line, want in cases:
|
||||
self.assertEqual(classify(f, p, line), want, f)
|
||||
|
||||
def test_target_repo(self):
|
||||
self.assertEqual(target_repo("/coulomb/key-cape.git"), "coulomb/key-cape")
|
||||
self.assertEqual(target_repo("/api/v1/repos/coulomb/state-hub/issues"), "coulomb/state-hub")
|
||||
self.assertEqual(target_repo("/v2/coulomb/flex-auth/manifests/x"), "coulomb/flex-auth")
|
||||
self.assertEqual(target_repo("/coulomb/activity-core@sha256:ab"), "coulomb/activity-core")
|
||||
self.assertIsNone(target_repo("/user/login"))
|
||||
|
||||
def test_runtime_anonymous_pull(self):
|
||||
rt = {"applications": [], "workloads": [{
|
||||
"namespace": "ns", "kind": "Deployment", "name": "api", "image_pull_secret_names": [],
|
||||
"images": ["forgejo.coulomb.social/coulomb/x@sha256:00"]}]}
|
||||
(c,) = runtime_consumers(rt)
|
||||
self.assertEqual(c["class"], "cluster-image-pull")
|
||||
self.assertTrue(c["credential"].startswith("anonymous"))
|
||||
self.assertEqual(c["target"], "coulomb/x")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
207
tools/access_consumers.py
Normal file
207
tools/access_consumers.py
Normal file
|
|
@ -0,0 +1,207 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Inventory everything that reads the forge without credentials (ORGREF-WP-0002-T02).
|
||||
|
||||
Read-only. Combines the runtime capture (ArgoCD sources and workload images)
|
||||
with a fresh scan of local working trees for forge URLs, classifies each hit
|
||||
by how it reads the forge, and groups the result into consumers: the repo or
|
||||
workload that reads, what it reads, and how. A consumer that reads over SSH
|
||||
with a personal key is recorded but is not anonymous.
|
||||
|
||||
python3 tools/access_consumers.py --root /home/worsch \
|
||||
--runtime inventory/runtime-2026-09-28.json \
|
||||
--output inventory/access-consumers-2026-10-08.json
|
||||
"""
|
||||
import argparse
|
||||
import collections
|
||||
import csv
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
FORGE_HOSTS = ("forgejo.coulomb.social", "gitea.coulomb.social")
|
||||
URL = re.compile(
|
||||
r"(?:https?://)?(?P<host>forgejo\.coulomb\.social|gitea\.coulomb\.social)"
|
||||
r"(?P<path>/[A-Za-z0-9._~%/@:+-]*)?")
|
||||
SKIP_DIRS = {".git", "node_modules", "vendor", "history", "archived", "__pycache__",
|
||||
".venv", "venv", "dist", "build", ".terraform", "rescued"}
|
||||
SKIP_FILES = {"WORK-RECORDS.md", ".custodian-brief.md"}
|
||||
MAX_BYTES = 1_000_000
|
||||
|
||||
# How a reference reads the forge. First match wins; order matters.
|
||||
KINDS = [
|
||||
("record", lambda f, p, line: f.startswith(("docs/evidence/", "evidence/", "tests/", "test/"))
|
||||
or "/evidence/" in f or "/tests/" in f or f.endswith(("PKG-INFO", ".sum"))),
|
||||
("raw-fetch", lambda f, p, line: "/raw/" in p or "/media/" in p),
|
||||
("api-client", lambda f, p, line: p.startswith("/api/")),
|
||||
("ci-action", lambda f, p, line: ".forgejo/workflows" in f or ".gitea/workflows" in f
|
||||
or ".github/workflows" in f),
|
||||
("package-consumer", lambda f, p, line: Path(f).name in {
|
||||
"go.mod", "go.sum", "pyproject.toml", "requirements.txt", "package.json",
|
||||
"uv.lock", "Cargo.toml", "setup.cfg"} or "/api/packages/" in p),
|
||||
("gitops-source", lambda f, p, line: "repoURL" in line or p.endswith(".git")
|
||||
and f.endswith((".yaml", ".yml"))),
|
||||
("image-pull", lambda f, p, line: bool(re.search(r"(image|repository|FROM)\b", line))
|
||||
or "@sha256:" in line or Path(f).name.startswith(("Dockerfile", "Containerfile"))),
|
||||
("doc-link", lambda f, p, line: f.endswith((".md", ".rst", ".txt", ".html"))),
|
||||
("build-tooling", lambda f, p, line: Path(f).name in {"Makefile", "justfile"}
|
||||
or f.endswith((".sh", ".mk"))),
|
||||
("code-reference", lambda f, p, line: f.endswith((".py", ".go", ".ts", ".js", ".rs"))),
|
||||
("config-reference", lambda f, p, line: f.endswith((".yaml", ".yml", ".json", ".toml",
|
||||
".ini", ".service", ".template", ".example"))),
|
||||
]
|
||||
|
||||
|
||||
def classify(file, path, line):
|
||||
for kind, test in KINDS:
|
||||
if test(file, path or "", line):
|
||||
return kind
|
||||
return "other"
|
||||
|
||||
|
||||
def target_repo(path):
|
||||
"""org/repo named by a forge URL path, if any."""
|
||||
parts = [p for p in (path or "").split("/") if p]
|
||||
if parts[:2] == ["api", "v1"] and len(parts) >= 5 and parts[2] == "repos":
|
||||
parts = parts[3:]
|
||||
elif parts and parts[0] in {"api", "v2"}:
|
||||
parts = parts[1:]
|
||||
if len(parts) >= 2 and parts[0] not in {"user", "explore", "assets"}:
|
||||
return f"{parts[0]}/{parts[1].removesuffix('.git').split('@')[0].split(':')[0]}"
|
||||
return None
|
||||
|
||||
|
||||
def scan_repo(repo):
|
||||
hits = []
|
||||
for dirpath, dirnames, filenames in os.walk(repo):
|
||||
dirnames[:] = [d for d in dirnames if d not in SKIP_DIRS and not d.startswith(".")
|
||||
or d in {".forgejo", ".gitea", ".github"}]
|
||||
for name in filenames:
|
||||
if name in SKIP_FILES:
|
||||
continue
|
||||
full = Path(dirpath) / name
|
||||
try:
|
||||
if full.is_symlink() or full.stat().st_size > MAX_BYTES:
|
||||
continue
|
||||
text = full.read_text(encoding="utf-8")
|
||||
except (OSError, UnicodeDecodeError):
|
||||
continue
|
||||
rel = str(full.relative_to(repo))
|
||||
for number, line in enumerate(text.splitlines(), 1):
|
||||
for m in URL.finditer(line):
|
||||
hits.append({
|
||||
"file": rel, "line": number, "host": m.group("host"),
|
||||
"kind": classify(rel, m.group("path") or "", line),
|
||||
"target": target_repo(m.group("path")),
|
||||
})
|
||||
return hits
|
||||
|
||||
|
||||
def remote_form(repo):
|
||||
try:
|
||||
url = subprocess.run(["git", "-C", str(repo), "remote", "get-url", "origin"],
|
||||
capture_output=True, text=True, check=True).stdout.strip()
|
||||
except (subprocess.CalledProcessError, OSError):
|
||||
return None
|
||||
if url.startswith(("forgejo-remote:", "ssh://", "git@")):
|
||||
return "ssh-key"
|
||||
if any(h in url for h in FORGE_HOSTS):
|
||||
return "https"
|
||||
return "other"
|
||||
|
||||
|
||||
def runtime_consumers(runtime):
|
||||
out = []
|
||||
for app in runtime.get("applications", []):
|
||||
for src in app.get("sources", []):
|
||||
u = src.get("repoURL", "")
|
||||
m = URL.search(u)
|
||||
if m:
|
||||
out.append({"consumer": f"argocd:{app['name']}", "class": "cluster-gitops",
|
||||
"kind": "gitops-source", "target": target_repo(m.group("path")),
|
||||
"credential": "unknown (ArgoCD repo credentials not inspected)"})
|
||||
for w in runtime.get("workloads", []):
|
||||
for image in w.get("images", []):
|
||||
m = URL.search(image)
|
||||
if m:
|
||||
secrets = w.get("image_pull_secret_names") or []
|
||||
out.append({"consumer": f"k3s:{w['namespace']}/{w['kind']}/{w['name']}",
|
||||
"class": "cluster-image-pull", "kind": "image-pull",
|
||||
"target": target_repo(m.group("path")),
|
||||
"credential": "pull secret named: " + ",".join(secrets)
|
||||
if secrets else "anonymous (no imagePullSecrets)"})
|
||||
return out
|
||||
|
||||
|
||||
CLASS_OF_KIND = {
|
||||
"raw-fetch": "raw-file-fetch", "api-client": "api-client", "ci-action": "ci-runner",
|
||||
"package-consumer": "package-consumer", "gitops-source": "repo-gitops",
|
||||
"image-pull": "repo-image-reference", "doc-link": "doc-link", "other": "other",
|
||||
"record": "record-or-test", "build-tooling": "build-tooling",
|
||||
"code-reference": "code-reference", "config-reference": "config-reference",
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
ap.add_argument("--root", default="/home/worsch")
|
||||
ap.add_argument("--runtime", required=True)
|
||||
ap.add_argument("--assignments", default="taxonomy/assignments.csv")
|
||||
ap.add_argument("--output", required=True)
|
||||
args = ap.parse_args()
|
||||
|
||||
stewards = {}
|
||||
with open(args.assignments, newline="") as fh:
|
||||
for row in csv.DictReader(fh):
|
||||
stewards[row["repository"]] = row["proposed_org"]
|
||||
|
||||
runtime = json.load(open(args.runtime))
|
||||
consumers = runtime_consumers(runtime)
|
||||
remotes = collections.Counter()
|
||||
scanned = []
|
||||
for repo in sorted(Path(args.root).iterdir()):
|
||||
if not (repo / ".git").exists() or repo.name.startswith("rescued-"):
|
||||
continue
|
||||
scanned.append(repo.name)
|
||||
remotes[remote_form(repo) or "none"] += 1
|
||||
grouped = collections.defaultdict(list)
|
||||
for h in scan_repo(repo):
|
||||
grouped[(h["kind"], h["target"])].append(f"{h['file']}:{h['line']}")
|
||||
for (kind, target), where in sorted(grouped.items(), key=lambda kv: (kv[0][0], str(kv[0][1]))):
|
||||
consumers.append({"consumer": f"repo:{repo.name}", "class": CLASS_OF_KIND[kind],
|
||||
"kind": kind, "target": target, "locations": where[:10],
|
||||
"location_count": len(where)})
|
||||
|
||||
for c in consumers:
|
||||
name = (c.get("target") or "").split("/")[-1]
|
||||
c["target_steward"] = stewards.get(name, "unassigned")
|
||||
|
||||
summary = collections.Counter(c["class"] for c in consumers)
|
||||
report = {
|
||||
"schema_version": "0.1",
|
||||
"workplan": "ORGREF-WP-0002-T02",
|
||||
"runtime_capture": os.path.basename(args.runtime),
|
||||
"scanned_repositories": len(scanned),
|
||||
"local_remote_forms": dict(remotes),
|
||||
"summary_by_class": dict(sorted(summary.items())),
|
||||
"anonymous_image_pulls": sum(1 for c in consumers if c["class"] == "cluster-image-pull"
|
||||
and c["credential"].startswith("anonymous")),
|
||||
"consumers": consumers,
|
||||
"limits": [
|
||||
"Runtime classes come from the runtime capture named above, not a live read.",
|
||||
"ArgoCD repository credentials and pull-secret contents were not inspected; "
|
||||
"a named pull secret is not proof that it authenticates to the forge.",
|
||||
"Only literal forge hostnames are found; computed URLs and consumers outside "
|
||||
"these working trees (external sites, Telegram, other people's clones) are not.",
|
||||
"history/ and archived/ directories are skipped; they record, they do not read.",
|
||||
],
|
||||
}
|
||||
Path(args.output).write_text(json.dumps(report, indent=1, sort_keys=False) + "\n")
|
||||
print(json.dumps({k: report[k] for k in ("scanned_repositories", "local_remote_forms",
|
||||
"summary_by_class", "anonymous_image_pulls")},
|
||||
indent=1))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
|
@ -122,7 +122,7 @@ Record a founder decision covering:
|
|||
|
||||
```task
|
||||
id: ORGREF-WP-0002-T02
|
||||
status: todo
|
||||
status: progress
|
||||
priority: high
|
||||
state_hub_task_id: "ace26a22-50e2-58c6-b4ad-85d95c284fce"
|
||||
```
|
||||
|
|
@ -146,6 +146,13 @@ Output a machine-readable `inventory/access-consumers-<date>.json` plus a
|
|||
summary listing, per consumer class, the owner and the proposed credential
|
||||
kind. Uncertain entries are flagged, not dropped.
|
||||
|
||||
2026-10-08: first pass done. `tools/access_consumers.py` produced
|
||||
`inventory/access-consumers-2026-10-08.json`, summarised in
|
||||
`inventory/access-consumers.md`. Headline: 40 cluster workloads pull forge images
|
||||
with no `imagePullSecrets`; 6 ArgoCD applications read forge Git; local clones
|
||||
are SSH and unaffected. Open: live check of node registry auth and ArgoCD
|
||||
credentials, and a per-entry review of api-client, code and CI hits.
|
||||
|
||||
```task
|
||||
id: ORGREF-WP-0002-T03
|
||||
status: todo
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue