Finish local collector preparation and block remaining coordination work

Assistant: codex
Assistant-Model: gpt-6.1-sol
Assistant-Session: 01a11beb-4eef-7fc1-94a7-597130a01ae9
This commit is contained in:
tegwick 2026-10-09 00:38:03 +02:00
parent a9af3b492f
commit 4a690f9e27
8 changed files with 291 additions and 23 deletions

View file

@ -47,7 +47,10 @@ dependency between the rollback prerequisite and T04. Preparation artifacts:
ORGREF-WP-0002 independently coordinates the accepted October 8 policy: private
forge repositories and packages, external deliberate publication, authenticated
consumers and staged visibility changes. T01–T03 have accepted preparation;
identity implementation and closure proofs remain with their owner records.
both workplans are blocked after local loose-end closure. Identity
implementation, private-read proof and closure receipts remain with their owner
records; collector authentication support is prepared without requesting any
credential. No remaining whole task can be accepted from local preparation alone.
See [policy](decisions/forge-visibility-policy.md) and the
[October 9 review](history/2026-10-09-review-and-maintenance.md). This work does
not satisfy or waive the four-condition org-transfer entry gate.

View file

@ -104,3 +104,9 @@ project's workplans, never copied.
- October 8–9: WP-0002 policy and inventories accepted; identity integration
waits on KEY-WP-0038 / NK-WP-0048 and RAPPS-WP-0016. Private forge access
is independent of the unchanged WP-0001 org-transfer gate.
- Loose-end closure October 9: both workplans are **blocked**. The collector's
authenticated read option is implemented and Telegram image upload delivery
is linked as owner-complete. Eleven remaining tasks retain their existing
evidence, owner and human gates; three WP-0002 tasks remain done. No new
tasks or workplans were opened.

View file

@ -67,3 +67,51 @@ preceding task, including the superseded forge owner. The checker updates
bodies but preserves these existing titles. Use file task IDs and the corrected
headings for meaning; state-hub owns the remaining title-projection repair.
This repo does not manually rewrite or re-register its task projection.
## Loose-end closure pass
The user requested completing what this repo can finish, retaining existing
work records and setting workplans with remaining blockers to blocked. Reviewed
both file-backed plans and the current State Hub summary: WP-0001 has five
waiting tasks; WP-0002 has three done and six waiting. There are no other
active, ready, proposed or blocked workplans for this repository in the current
summary. No remaining whole task has satisfied its acceptance criteria locally.
Completed local preparation within the existing tasks:
- WP-0001-T01 / WP-0002-T06: added explicit environment-variable read-token
support to the inventory collector, including private default-branch archives.
It is GET-only, confined to the current HTTPS forge origin, refuses credential
forwarding to another origin or HTTP, sends no forge token to State Hub, and
fails before collection if the requested credential variable is absent. Fixed
transport diagnostics contain no headers, response bodies or raw errors.
Authenticated output remains incomplete until owner attestation and the
existing coverage/acceptance requirements are met.
- WP-0002-T06: reconciled fluid-telegram FT-WP-0001-T14's completed upload and
founder-verified private-channel delivery. The owner record is linked by its
existing stable IDs. This proves upload delivery, not private forge reads;
credential provisioning/private-read proof is still outstanding.
- Removed the superseded WP-0001-T01 status-backfill instruction and completed
the policy-point enumeration; preserved all actual acceptance gates.
Both plans are now blocked. The existing tasks retain these blockers:
| Task(s) | Evidence needed before acceptance |
| --- | --- |
| WP-0001-T01 | Authorized complete repository/package/access export, exclusion/consumer disposition and unchanged-fleet paired inventory; decision 5f3680fc is still open |
| WP-0001-T02 | Accepted G1 and steward/maintainer/permission matrix; decision 60c5767c is still open |
| WP-0001-T03/T04 | All four entry conditions and applicable Git/package transfer rollback proof; gate-sequencing decision e9aca79b is still open |
| WP-0001-T05 | Accepted T01–T04 results before a cutover plan can start |
| WP-0002-T04 | KEY-WP-0038 registration and claims proof; registration depends on NK-WP-0048-T06 |
| WP-0002-T05 | RAPPS-WP-0016 break-glass, sign-in, account-linking and team-grant proof |
| WP-0002-T06 | Provisioned scoped read identities and accepted private repository/package/image receipts for every consumer |
| WP-0002-T07 | Consumer readiness and complete visibility/recovery rehearsal |
| WP-0002-T08 | Rehearsal and founder wave approval, or the already accepted interim route with all its guardrail receipts |
| WP-0002-T09 | Completed closure, anonymous-denial evidence and authenticated completeness/owner handback |
Validation: all 16 tests passed, including synthetic private metadata/archive
reads, recipient scoping, safe redirects/errors, absent-credential refusal and
unchanged anonymous pagination. Whitespace validation passed. No real token
was read or requested and no private forge proof was asserted. No owner tasks
were copied, no task/workplan was opened or cancelled, and no production state
was changed. The title-projection issue remains the existing state-hub residual.

View file

@ -35,6 +35,29 @@ paths; these are candidates for review, not all confirmed broken URLs. A referen
may have several surface categories. It does not detect arbitrary computed or
encoded paths. File location categories are triage hints, not semantic parsing.
## Authenticated collector preparation — completed locally, 2026-10-09
`tools/inventory.py --forge-token-env ORGREF_FORGE_READ_TOKEN` enables scoped
read authentication only when that named environment variable has been supplied
by the authorized credential route. Omit the flag for the anonymous baseline.
A requested missing, empty or invalid variable fails before network collection;
there is no silent anonymous fallback. Never put a credential value on the
command line, in a file, or in a receipt.
The same token is used for forge repository/package metadata and default-branch
archives. Requests are GET-only and confined to the exact current HTTPS forge
origin; cross-origin and HTTP downgrade redirects are refused. State Hub reads
remain separate and receive no forge token. Transport errors emit fixed messages
or HTTP status only, never request headers or response bodies. The snapshot
records `authentication: authenticated-read`, not the token or environment.
This option finishes local preparation within existing WP-0001-T01 /
WP-0002-T06. It does not grant a credential, waive the previous access denial,
attest the principal's scope, or turn `g1_complete` true. Actual authorized
private repository/package reads and owner completeness evidence remain pending.
The authenticated census also still needs access/automation exports, exclusion
and consumer disposition, and a matching unchanged-fleet pair.
## Live evidence
- `runtime-2026-09-28.json`: names/images/owner links for 132 workload controllers

View file

@ -1,8 +1,15 @@
import importlib.util
import io
import json
import os
from pathlib import Path
import subprocess
import tempfile
import unittest
from unittest.mock import Mock, patch
import urllib.error
import urllib.request
import zipfile
def module(name):
@ -48,4 +55,87 @@ class InventoryTests(unittest.TestCase):
m['assignments'][0]['proposed_org']='railiance'
self.assertFalse(verify.check(i,m)['clean'])
def test_authenticated_reads_use_get_and_are_scoped_to_forge(self):
response = Mock()
response.read.return_value = b'{"id": 42}'
opener = Mock()
opener.open.return_value.__enter__ = Mock(return_value=response)
opener.open.return_value.__exit__ = Mock(return_value=False)
with patch.object(inv.urllib.request, 'build_opener', return_value=opener):
self.assertEqual(inv.get(inv.BASE + '/api/v1/repos/coulomb/private', token='fixture-only'), {'id':42})
request = opener.open.call_args.args[0]
self.assertEqual(request.get_method(), 'GET')
self.assertEqual(request.get_header('Authorization'), 'token fixture-only')
for url in ['http://127.0.0.1:8000/repos/', 'https://other.invalid/repos/',
'http://forgejo.coulomb.social/repos/']:
with self.assertRaises(RuntimeError):
inv.get(url, token='fixture-only')
inv.get('http://127.0.0.1:8000/repos/')
self.assertIsNone(opener.open.call_args.args[0].get_header('Authorization'))
def test_authenticated_redirects_cannot_leak_token_or_downgrade(self):
req = urllib.request.Request(inv.BASE + '/start', headers={'Authorization':'token fixture-only'})
handler = inv.ScopedRedirects(True)
redirect = handler.redirect_request(req, None, 302, 'Found', {}, inv.BASE + '/next')
self.assertEqual(redirect.get_header('Authorization'), 'token fixture-only')
for url in ['https://other.invalid/next', 'http://forgejo.coulomb.social/next',
'https://user@forgejo.coulomb.social/next']:
with self.assertRaisesRegex(RuntimeError, '^Authenticated forge redirect refused$'):
handler.redirect_request(req, None, 302, 'Found', {}, url)
def test_transport_errors_do_not_emit_credential_or_raw_response(self):
for error, message in [
(urllib.error.URLError('fixture-only raw error'), 'Inventory read failed'),
(urllib.error.HTTPError(inv.BASE, 403, 'fixture-only raw response', {}, None), 'HTTP 403')]:
opener = Mock()
opener.open.side_effect = error
with patch.object(inv.urllib.request, 'build_opener', return_value=opener):
with self.assertRaises(RuntimeError) as caught:
inv.get(inv.BASE + '/api/v1/repos', token='fixture-only')
self.assertEqual(str(caught.exception), message)
def test_token_is_forwarded_to_all_metadata_reads_but_not_hub(self):
calls = []
def fake_get(url, *, token=None):
calls.append((url,token))
if url.endswith('/repos/'):
return []
if '/orgs/' in url and 'page=1' in url:
return [{'id':42, 'name':inv.SELF.name, 'full_name':'coulomb/'+inv.SELF.name, 'private':True}]
if '/orgs/' in url or '/packages/' in url:
return []
return {'version':'fixture'}
with tempfile.TemporaryDirectory() as root, patch.object(inv,'get',side_effect=fake_get):
result=inv.collect(Path(root),token='fixture-only')
self.assertFalse(result['g1_complete'])
self.assertEqual(result['forge']['authentication'],'authenticated-read')
self.assertTrue(result['server_repositories'][0]['private'])
for url,token in calls:
self.assertEqual(token, 'fixture-only' if url.startswith(inv.BASE) else None)
self.assertNotIn('fixture-only',json.dumps(result))
def test_missing_requested_credential_refuses_before_collection(self):
with tempfile.TemporaryDirectory() as root:
output=Path(root)/'snapshot.json'
env=dict(os.environ)
env.pop('ORGREF_TEST_MISSING_TOKEN',None)
result=subprocess.run([
'python3',str(Path(__file__).parents[1]/'tools/inventory.py'),
'--output',str(output),'--forge-token-env','ORGREF_TEST_MISSING_TOKEN'],
env=env,capture_output=True,text=True)
self.assertEqual(result.returncode,2)
self.assertFalse(output.exists())
def test_private_archive_uses_the_same_scoped_read_credential(self):
raw=io.BytesIO()
with zipfile.ZipFile(raw,'w') as archive:
archive.writestr('private/values.yaml','image: forgejo.example/coulomb/example:tag\n')
with patch.object(inv,'get',return_value={'commit':{'id':'abc'}}) as metadata, \
patch.object(inv,'read_bytes',return_value=raw.getvalue()) as archive_read:
result=inv.scan_archive({'name':'private','default_branch':'main'},token='fixture-only')
self.assertEqual(metadata.call_args.kwargs['token'],'fixture-only')
self.assertEqual(archive_read.call_args.kwargs,{'token':'fixture-only','limit':50_000_001})
self.assertEqual([r['reference'] for r in result['references']],['coulomb/example'])
self.assertNotIn('fixture-only',json.dumps(result))
if __name__=='__main__':unittest.main()

View file

@ -6,10 +6,12 @@ import hashlib
import io
import zipfile
import json
import os
import re
import subprocess
import urllib.error
import urllib.request
from urllib.parse import urlsplit
from pathlib import Path
BASE = 'https://forgejo.coulomb.social'
@ -27,19 +29,54 @@ def run(repo, *args):
return r.stdout
def get(url):
def origin(url):
parts = urlsplit(url)
return parts.scheme, parts.hostname, parts.port or (443 if parts.scheme == 'https' else 80)
class ScopedRedirects(urllib.request.HTTPRedirectHandler):
def __init__(self, authenticated):
self.authenticated = authenticated
def redirect_request(self, req, fp, code, msg, headers, newurl):
if self.authenticated and (origin(newurl) != origin(BASE)
or urlsplit(newurl).username is not None):
raise RuntimeError('Authenticated forge redirect refused')
return super().redirect_request(req, fp, code, msg, headers, newurl)
def read_bytes(url, *, token=None, limit=None):
headers = {}
if token is not None:
if not token or re.search(r'\s', token):
raise RuntimeError('Invalid forge read credential')
if origin(url) != origin(BASE) or urlsplit(url).username is not None:
raise RuntimeError('Forge credential destination refused')
headers['Authorization'] = 'token ' + token
request = urllib.request.Request(url, headers=headers, method='GET')
opener = urllib.request.build_opener(ScopedRedirects(token is not None))
try:
with urllib.request.urlopen(url, timeout=30) as r:
return json.load(r)
with opener.open(request, timeout=30) as response:
return response.read() if limit is None else response.read(limit)
except urllib.error.HTTPError as e:
raise RuntimeError('HTTP ' + str(e.code)) from None
except (urllib.error.URLError, OSError, ValueError):
raise RuntimeError('Inventory read failed') from None
def pages(path):
def get(url, *, token=None):
try:
return json.loads(read_bytes(url, token=token))
except (ValueError, UnicodeDecodeError):
raise RuntimeError('Invalid inventory JSON response') from None
def pages(path, *, token=None):
rows = []
seen = set()
for page in range(1, 1001):
batch = get(BASE + path + f'?limit=50&page={page}')
url = BASE + path + f'?limit=50&page={page}'
batch = get(url) if token is None else get(url, token=token)
if not isinstance(batch, list):
raise RuntimeError('expected paginated list')
if not batch:
@ -115,16 +152,15 @@ def scan(path):
def scan_archive(repo):
"""Read a public default-branch archive in memory, never extract to disk."""
def scan_archive(repo, *, token=None):
"""Read an accessible default-branch archive in memory, never extract to disk."""
name=repo['name']
branch=repo['default_branch']
from urllib.parse import quote
branch_info=get(BASE+'/api/v1/repos/coulomb/'+name+'/branches/'+quote(branch,safe=''))
branch_info=get(BASE+'/api/v1/repos/coulomb/'+name+'/branches/'+quote(branch,safe=''), token=token)
commit=branch_info['commit']['id']
url=BASE+'/api/v1/repos/coulomb/'+name+'/archive/'+commit+'.zip'
with urllib.request.urlopen(url,timeout=30) as response:
raw=response.read(50_000_001)
raw=read_bytes(url, token=token, limit=50_000_001)
if len(raw)>50_000_000:
raise RuntimeError('archive exceeds 50MB')
refs=[];exclusions=[]
@ -142,19 +178,21 @@ def scan_archive(repo):
return {'name':name,'commit':commit,'references':refs,'exclusions':exclusions}
def collect(root):
def collect(root, *, token=None):
gaps = [
'Anonymous forge enumeration cannot attest private repository/package completeness.',
('Anonymous forge enumeration cannot attest private repository/package completeness.'
if token is None else
'Authenticated enumeration is limited to the read principal grants; owner attestation of private/package completeness is still required.'),
'Team membership, deploy keys, webhooks, runners, protected branches and server-side access policies require an authorized owner export.',
'Local working trees and public remote-only default-branch archives are scanned; other branches and inaccessible repos are not content-scanned.',
'Local working trees and accessible remote-only default-branch archives are scanned; other branches and inaccessible repos are not content-scanned.',
'Hidden sibling directories, ignored files, submodule contents outside discovered repos, binary/large/encrypted files and symlinks are not scanned; exclusions are recorded.',
'Runtime image/source inventory is collected separately; source scan cannot attest all runtime or external consumers.',
'Dynamic/computed org or repository names require owner review; token scan finds literal coulomb/ references only.',
]
hub = get('http://127.0.0.1:8000/repos/')
repos = pages('/api/v1/orgs/coulomb/repos')
repos = pages('/api/v1/orgs/coulomb/repos', token=token)
try:
packages = pages('/api/v1/packages/coulomb')
packages = pages('/api/v1/packages/coulomb', token=token)
except (RuntimeError, OSError):
packages = []; gaps.append('Package API unavailable.')
paths = {p.resolve() for p in root.iterdir() if p.is_dir() and not p.name.startswith('.') and (p/'.git').exists()}
@ -169,9 +207,10 @@ def collect(root):
remote_scans=[]
for repo in server:
if repo['full_name'] not in local_names and repo['name']!=SELF.name:
try: remote_scans.append(scan_archive(repo))
try: remote_scans.append(scan_archive(repo, token=token))
except (OSError,RuntimeError,ValueError): gaps.append('Remote archive unavailable: '+repo['full_name'])
return {'schema_version':1,'forge':{'base_url':BASE,'version':get(BASE+'/api/v1/version'),'authentication':'anonymous'},
return {'schema_version':1,'forge':{'base_url':BASE,'version':get(BASE+'/api/v1/version', token=token),
'authentication':'authenticated-read' if token is not None else 'anonymous'},
'g1_complete':False, 'coverage_gaps':gaps,
'server_repositories':sorted(server,key=lambda r:r['id']),
'registered_repositories':sorted(registered,key=lambda r:r['slug']),
@ -189,8 +228,20 @@ def main():
ap.add_argument('--root',type=Path,default=Path('/home/worsch'))
ap.add_argument('--output',type=Path,required=True)
ap.add_argument('--compare',type=Path)
ap.add_argument('--forge-token-env', help='Name of an environment variable supplied by the authorized read-credential route; never a token value')
args=ap.parse_args()
result=collect(args.root)
token = None
if args.forge_token_env is not None:
if not re.fullmatch(r'[A-Za-z_][A-Za-z0-9_]*', args.forge_token_env):
ap.error('Invalid credential environment variable name')
token = os.environ.get(args.forge_token_env)
if not token or re.search(r'\s', token):
ap.error('Forge read credential environment variable is unset, empty or invalid')
try:
result=collect(args.root, token=token)
except (RuntimeError, OSError, ValueError):
print('Inventory collection failed; no accepted snapshot written')
return 1
encoded=json.dumps(result,sort_keys=True,indent=2)+'\n'
args.output.write_text(encoded)
print(json.dumps({'repos':len(result['server_repositories']), 'local_repos':len(result['local_repositories']),

View file

@ -9,7 +9,7 @@ flavor: implementation
owner: codex
topic_slug: railiance
created: "2026-08-11"
updated: "2026-10-05"
updated: "2026-10-09"
related_repos:
- railiance-forge
- railiance-platform
@ -113,7 +113,8 @@ blocking_reason: "The September 28 collector is not G1 acceptance. Authenticated
priority: high
state_hub_task_id: "4964bf8f-96eb-59f0-8a50-5ecb5c7dbc8f"
```
2026-09-29 (CUST-WP-0074-T05 backfill): this task was `wait` with no external dependency or human gate stated, which Task Wait Qualifiers v0.1 treats as unqualified; set to `todo`. Re-add `wait` with a qualifier if a real gate exists.
Historical September 29 wait-qualifier backfill is superseded by the October 5
explicit human/evidence gate above; T01 remains correctly waiting.
Build the re-runnable `coulomb/` reference inventory (G1). Must cover: git
@ -279,3 +280,14 @@ an org split with an unplanned host or forge-product migration.
- T05 produces the plan only. Follow-on file-backed workplans must cover migration
waves (G5/G6), durable convention publication (G7) and retirement evidence.
Create those after foundation acceptance; no migration task is implied done.
## Loose-end disposition — 2026-10-09
Remain blocked. T01's collector preparation now supports an explicitly supplied
scoped read credential, but the owner export/authorization decision, coverage
and unchanged-fleet evidence are still missing. T02 cannot be accepted until
G1 and the steward/permissions matrix are accepted. T03/T04 still wait on the
unchanged all-four gate and open rollback-sequencing decision; T05 requires
their accepted results. No remaining whole task is independently completable
here today, and no task is cancelled merely to make the plan look closed.
No new task or workplan was opened.

View file

@ -4,7 +4,7 @@ type: workplan
title: "Close the forge: NetKingdom access control first, then private by default"
domain: infotech
repo: prj-forgejo-org-refactor
status: active
status: blocked
flavor: implementation
owner: claude-code
topic_slug: railiance
@ -24,6 +24,15 @@ state_hub_workstream_id: "f873c10f-1408-5bf2-a834-fed88387804b"
# ORGREF-WP-0002 - Close the forge: NetKingdom access control first, then private by default
## Status — loose-end closure, 2026-10-09
Blocked at the user's request: T01–T03 are done, and T04–T09 have no remaining
independently executable acceptance work in this coordinator. The local
collector now supports an authorized scoped read token; actual provisioning,
private-read proof, identity integration and production wave evidence remain
external or human gates. Do not mark implementation or proof done from a draft,
a completed child task with narrower scope, or the existence of a token option.
## Why
`forgejo.coulomb.social` does not distinguish public-facing repositories from
@ -120,6 +129,8 @@ Record a founder decision covering:
5. **Interim lockdown:** whether to close a short list of sensitive repositories
now with existing local accounts. If yes: which ones, accepting that their
consumers move to local tokens temporarily and are re-homed in T06.
6. **Exposed history:** scans and owner remediation run in parallel with waves,
as accepted in the policy; T03 does not gate visibility waves.
2026-10-08: all six policy points and the driver role decided, recorded in
`decisions/forge-visibility-policy.md`. The forge is private with no anonymous
@ -279,7 +290,7 @@ their team grants. Break-glass login is verified once and recorded.
id: ORGREF-WP-0002-T06
status: wait
depends_on: [ORGREF-WP-0002-T02, ORGREF-WP-0002-T05]
blocking_reason: "Each consumer gets its credential from the T02 inventory, using the identities T05 makes available."
blocking_reason: "T05 identity/team proof and owner private-canary receipts remain missing. The local collector token option and FT-WP-0001-T14 upload delivery are implemented; authorized private forge access is not yet proven for every consumer."
priority: high
state_hub_task_id: "1285f94d-3b57-502b-8aeb-96fab9578287"
```
@ -301,6 +312,24 @@ this project, while anonymous access to that canary fails:
Each owner records its proof in its own workplan. This plan tracks the list.
2026-10-09 closure: the coordinator's `tools/inventory.py` accepts an explicitly
named environment variable supplied by the sanctioned credential route, scopes
the credential to GET requests on the current HTTPS forge origin, refuses
cross-origin/downgrade redirects and uses the same route for archive reads.
The collector never sends that token to State Hub and never persists it. Tests
prove these boundaries with synthetic credentials. No real credential was
requested or private forge probe performed, so T06 is still waiting.
Existing owner progress: fluid-telegram `FT-WP-0001-T14`
(`c822885b-c2a2-57a7-a448-7d1a853a26c0`, parent
`a660ed65-700e-5b54-8d91-a556b73518f0`) is done: signed test image upload was
verified by the founder on October 8 (private Telegram channel, message 4).
This closes the upload implementation/delivery handoff. It does not establish
that a provisioned read identity can fetch a private forge image, nor that
replacing an existing image is supported (the owner records caption-only edits).
Private forge access remains part of T06's owner acceptance; no new task is
opened here.
### T07 — Rehearse the flip and the rollback
```task
@ -382,3 +411,9 @@ Remaining coordination issue: the consistency sync corrects projected task
bodies but preserves existing shifted titles. Use the stable task IDs and file
headings above. Projection title repair belongs to state-hub; no task is
re-registered or renumbered here.
Loose-end disposition: all remaining tasks stay on their existing records.
T04/T05 wait for the linked identity owners; T06 waits for scoped identities
and private-canary evidence; T07 waits for those consumers; T08 waits for a
rehearsal and founder wave approval (or the approved interim route's full
receipts); T09 waits for completed closure. No new workplan or task is needed.