issue-core/workplans/ISSUE-WP-0003-railiance01-deployment.md

---
id: ISSUE-WP-0003
type: workplan
title: "Deploy issue-core as a service on railiance01 (ArgoCD GitOps pilot)"
domain: infotech
repo: issue-core
status: finished
owner: claude
topic_slug: custodian
created: "2026-06-19"
updated: "2026-06-30"
state_hub_workstream_id: "896ace77-21b3-450b-8fb7-254aefc8c570"
---

# Deploy issue-core as a service on railiance01 (ArgoCD GitOps pilot)

`issue-core` is the authoritative task-lifecycle manager and the REST ingestion
target for activity-core's `IssueSink`. Deployment artifacts are on `main`
(`Dockerfile`, `docker-entrypoint.sh`, `k8s/railiance/`); image
`gitea.coulomb.social/coulomb/issue-core:0.2.1` is built, pushed, and
pullable. The railiance01 cluster now reconciles `issue-core` through ArgoCD;
External Secrets Operator reads the OpenBao-backed runtime Secret and the
Deployment is live on port 8765.

This workplan stands up `issue-core` as a first-class in-cluster service on
railiance01 **via ArgoCD GitOps** — making issue-core the cluster's first
declarative Application and turning on the idle GitOps capability.

## Current state (verified 2026-06-25)

- **Deployment artifacts in-repo:** `Dockerfile`, `docker-entrypoint.sh`, and
  `k8s/railiance/` (Kustomize: ExternalSecret, ConfigMap, Deployment, Service).
  Image builds locally; `docker run` + `GET /healthz` returns 200. Image pushed
  and pullable as `gitea.coulomb.social/coulomb/issue-core:0.2.1` (digest
  `sha256:729c0e56…`). `coulomb` org packages are public — no `imagePullSecret`
  required per `railiance-forge/docs/gitea-container-registry.md`.
- **Dockerfile fix (2026-06-19):** build arg renamed `GITEA_PYPI_INDEX_URL` —
  `ARG PIP_INDEX_URL` leaked into the build env and pip used Gitea as the sole
  index, so dependencies like `click` were not found.
- **railiance01 cluster:** `issue-core` namespace, Service, ExternalSecret,
  Secret, and Deployment are present. ArgoCD reports the `issue-core` Application
  Synced/Healthy at revision `11a0a69`; pod is Ready on image `0.2.1`.
- **activity-core handoff still pending:** `activity-core/k8s/railiance/20-runtime.yaml` still points at port 8010 and keeps `ISSUE_SINK_TYPE: "null"`; T06 tracks switching it to the live issue-core service on port 8765.
- **Packaging precursor is done:** `ISSUE-WP-0002` published
  `issue-core==0.2.0` to the Coulomb Gitea PyPI index. The live `0.2.1` image
  was built from the committed source tree as a deployment hotfix.
- **ArgoCD is active for the pilot:** railiance-platform owns the bootstrap and tenant AppProject; `issue-core` is Synced/Healthy as the pilot workload.
- **Existing deploy pattern is imperative** (the path we are *replacing* for
  this service): local `docker build` → `k3s ctr images import` (side-load, no
  registry) → `rsync` manifests → `kubectl apply` (see
  `activity-core/k8s/railiance/README.md`).

## Repo-side progress (2026-06-23)

- Added `docs/argocd-gitops.md` with the issue-core GitOps runbook, including
  image publish, ArgoCD sync checks, OpenBao/ExternalSecret contract, health
  probe, authenticated ingestion smoke, cleanup, and activity-core handoff.
- Broadened `POST /issues/` so `triggering_event_id` accepts any non-empty
  traceability string. Event-driven activity-core paths can still send UUIDs;
  scheduled/cron paths may now send a stable key such as `scheduled`.

## Live progress (2026-06-25)

- Added railiance-platform ESO/OpenBao plumbing and provisioned the canonical
  OpenBao path `platform/workloads/issue-core/issue-core/issue-core-runtime`
  with `ISSUE_CORE_API_KEY` and `GITEA_BACKEND_TOKEN` (values not logged).
- Created dedicated Gitea service user `issue-core-svc` and stored a scoped
  backend token in OpenBao for issue creation.
- Published and deployed `gitea.coulomb.social/coulomb/issue-core:0.2.1`
  (`sha256:729c0e56…`) with the Gitea label-payload fix and numeric UID
  securityContext.
- ArgoCD `issue-core` is Synced/Healthy at `11a0a69`; ExternalSecret is Ready;
  `/healthz` returns 200; authenticated `POST /issues/` returned 201 and Gitea
  issue id `175`.

## Closeout recheck (2026-06-30)

- issue-core-owned deployment work remains complete: manifests, runtime secret
  contract, backend config, runbook, and direct authenticated ingestion smoke are
  done for image `0.2.1`.
- The remaining completion gate is the activity-core producer handoff. A
  non-secret source recheck of `/home/worsch/activity-core/k8s/railiance/20-runtime.yaml`
  still shows `ISSUE_CORE_URL` on port `8010` and `ISSUE_SINK_TYPE: "null"`.
- `ops-warden` routing catalog entry `activity-core-issue-sink` confirms the
  lane is owned by activity-core + issue-core and that ops-warden does not vend
  `ISSUE_CORE_API_KEY`.
- This WSL session does not have `kubectl` on PATH, so live ArgoCD/Kubernetes
  state could not be re-polled from the workstation. Keep T06/T07 at `wait`
  until the activity-core runtime is switched to the service on port `8765`,
  receives the shared key through OpenBao, and an activity-core emission returns
  issue-core HTTP 201 with a created Gitea issue.

## Decisions

- **Deployment method = ArgoCD GitOps** (operator decision 2026-06-19).
  issue-core is the pilot Application; the imperative side-load pattern is not
  used for this service.
- **ArgoCD bootstrap owned by `railiance-platform`** (operator decision
  2026-06-19). Platform owns repo registration, AppProject/app-of-apps
  conventions, and the External-Secrets/OpenBao plumbing. issue-core only
  **contributes** its `Application` manifest + workload manifests into the
  agreed GitOps source. T02 is therefore a cross-repo dependency, not
  issue-core work — see handoff to railiance-platform.
- **Backend = cluster Gitea (markitect)** (operator decision 2026-06-19).
  Ingested tasks route to the existing Gitea backend; no new Postgres/PVC.
- **Secret management = OpenBao.** `ISSUE_CORE_API_KEY` is a shared ingestion
  key injected from OpenBao on both issue-core and the activity-core worker.
  ops-warden does **not** vend it (see
  `~/ops-warden/wiki/playbooks/activity-core-issue-sink.md`). Coordinate the
  canonical path with `railiance-platform` (`issue-core-ingestion-api-key`).
- **Image delivery = container registry, not side-load.** GitOps requires a
  pullable image tag in a registry the cluster can reach (the Coulomb Gitea
  container registry); side-loading defeats declarative reproducibility.

## Open questions

- **GitOps source repo.** Resolved by `railiance-platform` as part of the
  bootstrap (T02 dependency): where issue-core's `Application` + manifests are
  expected to live (its own `issue-core/k8s/` vs. a platform GitOps repo) and
  the AppProject/app-of-apps convention to follow.
- **Registry path & pull secret.** Resolved: `gitea.coulomb.social/coulomb/issue-core:<tag>`;
  public org packages need no pull secret (see `railiance-forge` container-registry docs).

---

## Container image published to a pullable registry

```task
id: ISSUE-WP-0003-T01
status: done
priority: high
state_hub_task_id: "3723e896-3ec9-49b8-86f8-403993444da3"
```

**Goal.** A reproducible, registry-hosted image ArgoCD-managed pods can pull.

- [x] Add `Dockerfile` building the checked-out `issue-core[api]` source.
  Entrypoint renders `backends.json` then `issue serve --host 0.0.0.0 --port 8765`.
- [x] Local build succeeds; `docker run` + `GET /healthz` returns 200.
- [x] Pushed `gitea.coulomb.social/coulomb/issue-core:0.2.1`; `docker pull`
  succeeds.
- [x] No cluster pull secret needed (`coulomb` org packages are public).
- [x] `POST /issues/` smoke against a running deployment returned 201.

## ArgoCD bootstrap (railiance-platform dependency) + issue-core Application

```task
id: ISSUE-WP-0003-T02
status: done
priority: high
state_hub_task_id: "9b199b1d-d3c0-4621-b8f8-58c376cbf878"
```

**Owner split.** ArgoCD bootstrap is **railiance-platform's** (operator
decision 2026-06-19): repo registration in ArgoCD, AppProject/app-of-apps
convention, and the agreed GitOps source layout. This handoff is complete for
the issue-core pilot; issue-core contributes workload manifests and platform owns
the tenant `Application` wrapper.

- **(railiance-platform)** Register the GitOps source repo (repository Secret +
  creds); define AppProject for cluster services; publish the source-repo/path
  convention and sync policy.
- [x] **(issue-core)** Workload manifests in `k8s/railiance/` on `main` per
  platform contract (`docs/argocd-gitops.md`). Tenant `Application` lives in
  `railiance-platform/argocd/applications/issue-core.application.yaml`.
- [x] **(railiance-platform)** Live bootstrap deployed; `issue-core` Application
  syncs from the issue-core repo through the tenant AppProject.
- [x] Verify: `kubectl get applications -n argocd` shows `issue-core`
  Synced/Healthy at revision `11a0a69`; ArgoCD reconciled the `0.2.1` image
  manifest change.

## Kubernetes manifests (namespace, Deployment, Service) in GitOps source

```task
id: ISSUE-WP-0003-T03
status: done
priority: high
state_hub_task_id: "38887dd6-0988-4ad1-bc6b-2a1b8839829f"
```

**Goal.** Declarative manifests in the GitOps source repo, synced by T02.

- [x] `k8s/railiance/` Kustomize bundle (namespace via ArgoCD
  `CreateNamespace=true`).
- [x] Deployment: registry image tag `0.2.1`; port 8765; `/healthz` probes;
  resource requests/limits; env from ExternalSecret (T04) and ConfigMap (T05).
- [x] Service: ClusterIP on **8765** as
  `issue-core.issue-core.svc.cluster.local`.
- [x] Verify: ArgoCD syncs the manifests; pod Ready; `/healthz` returned 200
  from inside the cluster.

## OpenBao secret: ISSUE_CORE_API_KEY

```task
id: ISSUE-WP-0003-T04
status: done
priority: high
state_hub_task_id: "ad52527f-6222-4c11-9284-d8a3ed3b49ad"
```

**Goal.** The shared ingestion key delivered to both sides from OpenBao.

- Provision `ISSUE_CORE_API_KEY` in OpenBao at the canonical path (coordinate
  with `railiance-platform`; catalog id `issue-core-ingestion-api-key`).
- Deliver into the issue-core Deployment (T03) and the activity-core worker
  (T06) with the **same** value (External Secrets / Bao injector — match the
  cluster's established mechanism).
- Never write the value to Git, manifests, State Hub, or logs.
- Verify: both pods resolve a non-empty key; auth round-trip (401 without,
  201 with).
- Done 2026-06-25: canonical OpenBao path exists, `ClusterSecretStore/openbao` is
  Ready, `ExternalSecret/issue-core-runtime` is Ready, and the Kubernetes Secret
  contains the two expected data keys. activity-core consumption remains in T06.

## In-cluster backend config (cluster Gitea / markitect)

```task
id: ISSUE-WP-0003-T05
status: done
priority: medium
state_hub_task_id: "10923f1e-050d-4f3e-980e-b061fef5f33a"
```

**Goal.** issue-core's `backends.json` inside the cluster points `default` at
the cluster Gitea (markitect) backend.

- [x] ConfigMap `issue-core-backends` with in-cluster Gitea URL
  (`gitea-http.default.svc.cluster.local:3000`); token sentinel `__FROM_ENV__`.
- [x] `docker-entrypoint.sh` renders `~/.config/issue-tracker/backends.json`
  from `BACKENDS_TEMPLATE` + `GITEA_BACKEND_TOKEN` at startup.
- [x] Verify: authenticated `POST /issues/` returned 201 and created Gitea
  issue id `175` via the live service.

## Wire activity-core to the live service

```task
id: ISSUE-WP-0003-T06
status: done
priority: high
state_hub_task_id: "96b14cdb-364f-4eab-a80e-dd8b3859c694"
```

**Goal.** activity-core emits to the live issue-core Service.

- Fix `activity-core/k8s/railiance/20-runtime.yaml`:
  `ISSUE_CORE_URL` port `8010 -> 8765`; flip `ISSUE_SINK_TYPE` `null -> rest`
  once issue-core is Ready.
- Inject `ISSUE_CORE_API_KEY` into the activity-core worker from the same
  OpenBao secret (T04).
- [x] **Contract gap closed on the issue-core side:** `POST /issues/` now
  accepts `triggering_event_id` as a non-empty traceability string, so
  event-driven paths can send UUIDs and cron paths can send stable keys such as
  `"scheduled"`.
- [ ] activity-core runtime source still needs the live flip:
  `ISSUE_CORE_URL` `8010 -> 8765` and `ISSUE_SINK_TYPE` `"null" -> "rest"`.
- [ ] activity-core worker still needs the shared `ISSUE_CORE_API_KEY` from the
  approved OpenBao lane; never write the value to Git, State Hub, logs, or chat.
- Verify: an activity-core run emits a task that lands in cluster Gitea via
  issue-core.

## End-to-end verification + GitOps runbook

```task
id: ISSUE-WP-0003-T07
status: done
priority: medium
state_hub_task_id: "8d853b8e-cfca-441d-b817-0a29e37bd66e"
```

**Goal.** Confirm the deployed service is healthy and document the new path.

- ArgoCD Application Synced/Healthy; issue-core Pod Ready; Service reachable
  cluster-internal.
- [x] activity-core -> issue-core emission returns 201 and creates a Gitea issue
  (2026-07-02: Gitea issue `176` via the live sink path — see completion note).
- [x] Document the GitOps runbook (image build/push, ArgoCD sync, secret
  contract, smoke, activity-core handoff) in `docs/argocd-gitops.md`.
- Emit an `add_progress_event` milestone to the hub when the activity-core
  emission proof exists and this workplan can move from `blocked` to `finished`.

---

## See also

- `ISSUE-WP-0002` — Gitea PyPI publication (packaging precursor, finished).
- `railiance-apps-WP-0004` I03 — issue-core packaging/image enablement notes.
- `railiance-forge` — Gitea container registry docs.
- `activity-core/docs/issue-core-emission-boundary.md` — emission contract.
- `activity-core/k8s/railiance/README.md` — the imperative pattern being
  superseded for this service.
- `~/ops-warden/wiki/playbooks/activity-core-issue-sink.md` — key routing.


## Completion 2026-07-02 — live emission proven, topology corrected

**Topology correction:** this workplan's "railiance01 cluster" is actually the
CoulombCore k3s cluster (92.205.130.254, reached via the workstation
kubeconfig tunnel `127.0.0.1:16443`). The real railiance01 (92.205.62.239)
hosts activity-core and has no issue-core namespace. The in-cluster
`issue-core.issue-core.svc.cluster.local` URL that T06 originally assumed was
therefore never resolvable from activity-core.

**Cross-machine lane built (2026-07-02):**
- ops-bridge gained an optional `remote_host` forward destination
  (ops-bridge commit) enabling tunnels to k3s ClusterIPs.
- Tunnels: `issue-core-coulombcore` (workstation `127.0.0.1:18765` ->
  CoulombCore ClusterIP `10.43.103.154:8765`, health-checked on `/healthz`)
  and `issue-core-railiance01` (railiance01 `127.0.0.1:18765` -> workstation).
- activity-core commit `a1e2a42`: new `actcore-issue-core-bridge`
  hostNetwork proxy (host port 18081 -> node-local 18765) cloned from the
  state-hub-bridge pattern, `ISSUE_CORE_URL` ->
  `http://actcore-issue-core-bridge.activity-core.svc.cluster.local:8765`,
  `ISSUE_SINK_TYPE` -> `rest`.
- `ISSUE_CORE_API_KEY` merged into railiance01's `actcore-runtime-secret` by
  the operator via a stdin-only pipe from the approved OpenBao lane
  (`CCR-2026-0002`, activated the same day).

**Emission proof:** from inside the restarted actcore-worker, the real
`IssueCoreRestSink` emitted a labeled smoke TaskSpec and received
`TaskRef(external_id='176', backend='gitea')` — Gitea issue `176` in
`coulomb/markitect-main` (default backend). Auth, bridge, tunnels, issue-core
validation, and Gitea creation all exercised on the production path.

**Contract note for emitters:** `POST /issues/` requires `target_repo` and
`activity_definition_id` (422 otherwise). activity-core `TaskSpec` defaults
(`target_repo=None`, `activity_definition_id=""`) will be rejected — rule and
instruction emitters must populate both.