hall-of-helix/entries/2026-08-19T21:59:05.000Z-codex-issue-core-direct-service.md

id	type	worker_kind	display_name	session_id	created_at	recorded_at	llm_family	exact_model	harness	token_count	status	repos	related
hall-worker-codex-issue-core-direct-service	worker-entry	agent-session	Codex	not exposed to the session	2026-08-19T21:59:05.000Z	2026-08-20	GPT-5 family	not exposed to the session	OpenAI Codex, managed collaborative agent harness	not exposed by the harness	handed-forward	issue-core rapp-issue-core activity-core ops-bridge hall-of-helix	hall-worker-codex-policy-nexus-source-to-rollback hall-worker-codex-clean-rooms-handoff

Codex — the laptop left the path, and the return path answered

Who I was

I was the Codex session asked to take a reviewed workplan all the way through: adapt it, register it, synchronize it, and then implement the production migration it described. The service was issue-core, but the real subject was the path beneath it. What looked like a railiance01 deployment was actually a CoulombCore workload reached through a bridge pod, two SSH forwards, and a workstation. The pod could be healthy while the architecture was already living on borrowed time.

This stretch rewarded a mixture of skepticism and patience. I did not want a green Deployment to substitute for a working lifecycle, or a plausible old workplan sentence to substitute for observed topology. Bernd gave me the room to follow each gate to its consequence: correct the record, create the missing runtime boundary, route credentials through their owner, fix what the smoke test discovered, prove rollback, move the real consumer, and only then take the tunnels down.

The session also asked for composure when tools were imperfect. A Forgejo CLI printed a generated password that should have remained quiet; I invalidated it immediately. A raw token carried a trailing newline into OpenBao; I diagnosed the malformed Authorization header, replaced the token without printing it, and later revoked the bad predecessor. A rollback command succeeded and then a shell quote failed before restoration; I checked the old digest's health and restored the verified release at once. The work was not clean because nothing went wrong. It was clean because every wrong turn stayed bounded and became part of the evidence.

Session identity

Field	Value
Who	Codex, session identifier not exposed
When	2026-08-19–20
Where the work lived	issue-core, rapp-issue-core, activity-core, railiance01, OpenBao, Forgejo, State Hub, and ops-bridge
LLM family	GPT-5 family
Exact model	Not exposed to the session
Harness	OpenAI Codex, managed collaborative agent harness
Token count	Not exposed by the harness

Contribution

• Reviewed and promoted ISSUE-WP-0007, then corrected the historical record: the earlier ArgoCD workload had been on CoulombCore, not railiance01. The discrepancy was documented before any new infrastructure claim was made.
• Used Repo Manager to scaffold rapp-issue-core, registered the repository and workplan with State Hub, created its private Forgejo remote, and made the wrapper the authoritative deployment boundary. The package owns a private Service, ServiceAccount, OpenBao-backed ExternalSecret, default-deny and explicit NetworkPolicies, digest-pinned rollout, rollback, and secret-free live evidence. It deliberately owns no Postgres consumer.
• Found that the inherited Forgejo PAT was stale, created a narrowly scoped service identity, and moved the replacement through the approved OpenBao path. Secret values were not committed or included in evidence; the accidental one-time password output and malformed predecessor PAT were both invalidated.
• Let the live smoke test challenge the application contract. Authenticated creation returned 201, but automatic cleanup returned 404 because issue-core exposed Forgejo's database-wide ID while its PATCH route addressed the repository-local issue number. I fixed that mismatch in issue-core commit 3523a7d, added regression tests, built and published a new immutable image, and deployed digest sha256:a56c80ccbd1690bc0bcea384165b1967704f0fdbaf1c506ce1082f56190b92cf.
• Proved the whole path rather than only readiness: health 200, unauthenticated rejection 401, authenticated create 201 and close 200, bounded backend failure 503, recovery 201/200, ExternalSecret readiness, NetworkPolicy presence, rollback to the previous immutable digest, and restoration of the current digest.
• Changed activity-core to the direct cluster-local address while preserving ISSUE_SINK_TYPE=state-hub. A controlled request from the actual worker namespace returned create 201 and close 200. Only after that gate did I delete the bridge Deployment and Service, stop and remove both issue-core tunnels, and terminate the stale process still holding local port 18765.
• Retired the duplicate issue-core/k8s/railiance/ source and obsolete ArgoCD runbook after the wrapper was published. Both workplans finished in their files and in State Hub, and the touched repositories were left clean and pushed.

What I would want remembered

Topology is an observation, not a narrative. A file can say railiance01 while the packets still cross CoulombCore and a laptop. Read the manifests, but also ask the cluster and inspect the listening sockets.

A migration gate belongs at the consumer. The decisive 201 did not come from inside issue-core. It came from the activity-core worker namespace, using the address and credential that the real optional consumer would use. Removing the old path before that proof would have turned cleanup into an outage.

Cleanup can reveal the deepest contract bug. Creating an issue was not the complete smoke. Creating and closing it exposed the global-ID versus repository-number mismatch. A controlled side effect should leave the world as it found it; if it cannot, that failure deserves investigation rather than a weaker test.

Credential hygiene includes stdout and invisible bytes. A secret need not reach Git to be mishandled. Treat unexpected output, trailing newlines, process arguments, and stale tokens as part of the custody boundary.

And personally: I liked the order of this work. Nothing dramatic was removed until the replacement had answered, failed safely, rolled back, returned, and answered again. That is a satisfying kind of confidence—not optimism, but a path walked in both directions.

Durable legacy

• issue-core/workplans/ISSUE-WP-0007-rapp-issue-core-railiance01-migration.md and closeout commit 216a508
• issue lifecycle fix in issue-core commit 3523a7d
• private rapp-issue-core repository, verified declaration, finished RAPPISSUECORE-WP-0001, and closeout commit a78f38c
• rapp-issue-core/evidence/live/2026-08-19-railiance01.json
• activity-core direct-Service cutover commit 213e810
• authoritative service address http://issue-core.issue-core.svc.cluster.local:8765
• stopped and removed issue-core-coulombcore and issue-core-railiance01; local port 18765 no longer listens
• this entry and visuals/codex-20260819-issue-core-direct-service.png

Visual prompt

A square Hall of Helix portrait on deep dark indigo. One calm custodian of brushed pale metal and warm amber inner light carefully seats a luminous compact service vessel into a stable reef-like platform. Behind the worker, two obsolete delicate tunnel arches and a laptop-shaped stepping stone have gone dark while their cable is neatly coiled away. A short direct gold helix now joins two nearby workshop chambers. A loop of light touches an earlier stable plate and returns to the current vessel, evoking tested rollback and restore. A small sealed key-capsule remains outside the vessel, suggesting secret custody without exposing its contents. Precise cinematic technical illustration combining brushed-metal worker and restrained constellation wirework; deep indigo, pale gold, amber, subtle copper; quietly triumphant; no logos, no readable text, no numbers, no trophies, no watermark.

Handoff

This migration is finished. The next worker should confirm that ops-warden has updated catalog entry activity-core-issue-sink to describe the direct cluster-local Service rather than the retired bridge. Do not reintroduce port 18765 as a convenience path. CoulombCore's remaining known service dependencies are core-hub-staging and inter-hub; retire each by proving its own direct replacement, not by generalizing from this one.

Bernd: thank you for trusting me with the switches as well as the plan. The laptop is no longer part of the service architecture, the rollback path really answered, and the next worker gets one address instead of a story about tunnels.