inter-hub-haskell/workplans/IHUB-WP-0016-build-infrastructure-and-error-loop.md

223 lines
7.9 KiB
Markdown
Raw Normal View History

---
id: IHUB-WP-0016
type: workplan
title: "Build Infrastructure: Incremental Compilation and Autonomous Error-Fix Loop"
domain: inter_hub
repo: inter-hub
status: done
owner: custodian
topic_slug: inter_hub
created: "2026-04-10"
updated: "2026-04-10"
depends_on: []
2026-04-25 17:03:52 +02:00
state_hub_workstream_id: "5dcacb24-f4a2-407c-a526-d39ef9d4a51a"
---
# IHUB-WP-0016 — Build Infrastructure: Incremental Compilation and Autonomous Error-Fix Loop
## Goal
Make GHC compilation fast enough for iterative error fixing without blocking the
machine or requiring user involvement. The first full build is unavoidably slow
(Nix dep download + all 180+ modules), but subsequent rebuilds must be incremental
(seconds per changed file). An autonomous error-fix loop must allow Claude to iterate
on compilation errors independently.
## Background
### Why compilation is slow
IHP compiles the entire project as one GHC executable target containing ~180 Haskell
modules (Generated types + Web.Types + 12 helpers + 59 controllers + 120 views +
FrontController/Routes). GHC compiles modules sequentially (`.ghci` enforces `-j1`
for this constrained host) and caches each module's `.o`/`.hi` files. Once a module
is compiled and its dependencies have not changed, it is NOT recompiled.
**First build**: slow — GHC compiles all 180 modules from scratch, plus Nix fetches
deps. Unavoidable. Expected: 2060 min on constrained hardware.
**Subsequent incremental builds**: fast — only changed modules and their dependents
recompile. A single controller change: ~530 seconds. This is what we rely on.
### Module dependency layers (compilation order, bottom-up)
```
Layer 1 (stable core — compile once, never touch during error loops):
build/Generated/*.hs IHP auto-generated from Schema.sql
Web/Types.hs 500-line controller type definitions
Layer 2 (helpers — only touch if Layer 1 is clean):
Application/Helper/*.hs 12 files
Generated/ (IHP code-gen output, if present)
Layer 3 (working surface — most errors will be here):
Web/Controller/*.hs 59 controllers
Web/View/**/*.hs 120 views
Layer 4 (wiring — fix last):
Web/FrontController.hs
Web/Routes.hs
```
**Rule**: never modify Layer 1 during error-fix loops. Changes to Web/Types.hs or
Generated types invalidate all 59 controllers and 120 views simultaneously.
### Why `devenv up` is overkill for error checking
`devenv up` starts: PostgreSQL, file-change watcher, Tailwind CSS watcher, AND
ghcid. For compilation error fixing we only need ghcid. Running ghcid standalone
skips the database startup, port binding, and service orchestration overhead.
## Tasks
### C1 — Create `scripts/compile-check` script ✓
```task
id: C1
status: done
priority: high
2026-04-25 17:03:52 +02:00
state_hub_task_id: "4c812cfa-8204-4714-8493-a29529f605ea"
```
Also created `scripts/compile-check-core` and `.ghci-core` to compile Layer 1+2
in isolation (no Main.hs, no Layer 3). Resolves the architecture gap where Layer
2 correctness could not be verified independently.
A shell script at `scripts/compile-check` that runs ghcid in isolation (no postgres, no tailwind):
```bash
#!/usr/bin/env bash
# scripts/compile-check
# Run ghcid standalone for fast compilation feedback.
# Outputs errors to stdout AND to /tmp/ihub-compile-errors.txt for monitoring.
# Does NOT start postgres or tailwind.
#
# Usage:
# scripts/compile-check # interactive, shows errors in terminal
# scripts/compile-check --bg # write to log only (for Claude monitoring)
#
# Pre-requisite: be inside `devenv shell` or have IHP_LIB set.
set -euo pipefail
LOGFILE="${IHUB_COMPILE_LOG:-/tmp/ihub-compile-errors.txt}"
: > "$LOGFILE" # truncate on start
echo "[compile-check] Writing errors to $LOGFILE"
exec ghcid \
--no-title \
--outputfile "$LOGFILE" \
--command "ghci"
```
This relies on `.ghci` (already present) which sets up `-j1` and loads the IHP config.
### C2 — Verify GHC object cache persistence
```task
id: C2
status: done
priority: medium
2026-04-25 17:03:52 +02:00
state_hub_task_id: "18d4d383-5f66-4780-a066-f25e51ccb158"
```
**Finding (2026-04-10):** IHP's `applicationGhciConfig` sets `-fbyte-code`. GHCi compiles
modules to bytecode in memory — there are **no persistent `.o` object files**. Each ghcid
session recompiles all bytecode from source.
Our `-fwrite-interface` flag (C4) writes `.hi` interface files alongside source modules.
These survive session restarts and allow GHCi to skip re-type-checking unchanged modules.
This saves some time on restarts but does NOT eliminate full bytecode recompilation.
**Net result:** No persistent `.o` cache. `.hi` files provide partial speedup (skip
type-checking). A single-module change still triggers full bytecode regeneration of that
module and its dependents. For true persistent incremental builds, `-fobject-code` would
be needed — not worth adding given IHP's design tradeoffs.
**Action:** No configuration change needed. Expectation documented here and in CLAUDE.md.
### C3 — Autonomous error-fix loop (ralph-workplan)
```task
id: C3
status: done
priority: high
2026-04-25 17:03:52 +02:00
state_hub_task_id: "56c45c72-1978-48c4-98ec-15d169c4417b"
```
Delivered as `workplans/IHUB-WP-0017-error-fix-loop.md`. Run inside `devenv shell`:
```bash
/ralph-workplan workplans/IHUB-WP-0017-error-fix-loop.md
```
Configure a ralph-workplan loop:
1. `scripts/compile-check --bg` runs ghcid in background, writing to `/tmp/ihub-compile-errors.txt`
2. Monitor tool watches the log file for new error output
3. On error output: read errors → identify failing module → apply fix → ghcid auto-reloads
4. On "All good" output: loop exits, reports clean build
**Loop discipline**:
- Fix errors bottom-up (Layer 1 → Layer 2 → Layer 3 → Layer 4)
- Fix one module at a time; wait for ghcid reload before fixing the next
- Do NOT make speculative changes to stable layers while fixing working-surface errors
- Maximum 3 fix attempts per module before escalating to user
### C4 — Add GHC `-fwrite-interface` and `-keep-going` flags to `.ghci` ✓
```task
id: C4
status: done
priority: high
2026-04-25 17:03:52 +02:00
state_hub_task_id: "5660575a-7fe4-471e-800f-a256964a0301"
```
```
-- Continue past errors in other modules (report all errors in one pass)
:set -fkeep-going
-- Ensure interface files are written even on partial success
:set -fwrite-interface
```
`-fkeep-going` lets GHC report errors from all modules in one pass rather than
stopping at the first failure — critical for batch error fixing.
### C5 — Document module surface constraints in CLAUDE.md ✓
```task
id: C5
status: done
priority: medium
2026-04-25 17:03:52 +02:00
state_hub_task_id: "243e1469-89e0-4966-a50f-33279a3be34f"
```
Added "Compilation Layers" section to CLAUDE.md covering:
- Never modify Layer 1 during error-fix loops (and why — cascade invalidation)
- Layer breakdown with file counts
- Each layer's expected compile time after first build
- Cache location and how to verify it's warm
- References to `scripts/compile-check` and `scripts/compile-check-core`
## Error-Fix Loop SOP (Standard Operating Procedure)
```
1. Ensure devenv shell is active (IHP_LIB must be set)
2. Run: scripts/compile-check --bg
3. Monitor /tmp/ihub-compile-errors.txt
4. Parse errors: group by module, order by Layer (1 → 4)
5. Fix Layer 1 errors first (rare; usually type/import issues in Web/Types.hs)
6. For each failing module in Layer 23:
a. Read current module state
b. Apply targeted fix
c. Wait for ghcid "Reloading..." + result
d. If still failing: re-read error, apply next fix
e. If 3 attempts fail: note module name, move on
7. After Layer 3 clean: fix Layer 4 (FrontController/Routes)
8. "All good (N modules)" in log = build clean
9. Commit fixes, update WP-0014/A1 status
```
## Exit Criteria
- `scripts/compile-check` script exists and runnable inside devenv shell
- `ghcid` produces output to `/tmp/ihub-compile-errors.txt` successfully
- GHC object cache location identified and documented
- ralph-workplan loop configuration in place
- Build reaches "All good" state without user intervention beyond starting the loop