fluid-core/cmd/fluid-control/main.go

193 lines
5.7 KiB
Go
Raw Normal View History

// Command fluid-control serves the FLUID control-plane APIs.
//
// It publishes revisions and records governance decisions. It is not on the
// request path: ArchitectureBlueprint.md invariant 2 requires the data plane to
// keep serving when this process is down, and that separation is only real if
// nothing here is reachable from the gateway.
//
// This surface must never be exposed to interface consumers. It is the
// mechanism that evolves the interface in response to their behaviour, and a
// consumer able to reach it could drive its own adaptation.
package main
import (
"context"
"crypto/ed25519"
"encoding/base64"
"errors"
"flag"
"fmt"
"log"
"net/http"
"os"
"os/signal"
"syscall"
"time"
_ "modernc.org/sqlite"
"github.com/tegwick/fluid-core/internal/contract"
"github.com/tegwick/fluid-core/internal/control"
"github.com/tegwick/fluid-core/internal/evidence"
"github.com/tegwick/fluid-core/internal/intent"
Add telemetry ingest, feedback collector, pressure API and insight CLI Completes FLUID-WP-0005. Normalization and redaction live on one path, shared by the in-process emitter and the ingest endpoint: two paths with two normalizations would eventually disagree, and the disagreement would surface as a pressure finding that is really a pipeline bug. Telemetry kind is inferred from event shape rather than defaulting to "request", since an error filed as a request understates the interface's failure rate. A malformed event in a batch does not discard the rest. Feedback is stored as evidence and creates no pressure and no hypothesis on its own, per API Standards 15, with the consumer recorded as the actor so their untrusted status stays visible in the audit trail. The feedback endpoint is the only consumer-reachable part of the control plane. The observation endpoints are not served at all when no pseudonymization salt is configured, rather than served with a generated one: a salt that changed per run would make the same consumer look new every time and every cohort count wrong. Adds an end-to-end test driving real traffic through the gateway and confirming it becomes a classified pressure record, with no raw consumer identity reaching the store. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014KmVxhJ35tCo7rE7UnLwWu Assistant: claude-code Assistant-Model: opus Assistant-Process: 1116572@bnt-lap001 Assistant-Session: 8ba9bb93-a72a-4883-b189-2499cce5c400
2026-09-04 03:19:06 +02:00
"github.com/tegwick/fluid-core/internal/observation"
"github.com/tegwick/fluid-core/internal/policy"
"github.com/tegwick/fluid-core/internal/publish"
Add science control APIs and the hypothesis, experiment and promote CLI Completes FLUID-WP-0006. The loop now runs end to end from the command line: two competing presentation hypotheses, an experiment that issues a routing policy rather than touching traffic, an amendment, a stop that returns traffic to the default, a confirmed outcome, a resolved competition, and a promotion the gate can refuse. Starting or stopping an experiment returns the routing policy for the operator to install rather than installing it. Blueprint 17 keeps the controller out of the traffic path, and installing from the handler would put it straight back in; emitting the document keeps the separation visible instead of implied. `fluid audit trace` now answers the section 25 questions from events rather than summary records, and names the rivals a hypothesis beat: an audit asking which hypotheses were considered is not answered by naming only the winner. Two fixes found by driving the CLI rather than only the tests. Go's flag package stops at the first positional, so ids given after flags silently swallowed them; ids are now taken before parsing. And there was no way to attach a revision to the hypothesis that produced it, which left `audit trace` unable to say why a revision existed -- `hypothesis attach` closes that. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014KmVxhJ35tCo7rE7UnLwWu Assistant: claude-code Assistant-Model: opus Assistant-Process: 1116572@bnt-lap001 Assistant-Session: 8ba9bb93-a72a-4883-b189-2499cce5c400
2026-09-04 06:44:47 +02:00
"github.com/tegwick/fluid-core/internal/science"
"github.com/tegwick/fluid-core/internal/signing"
)
func main() {
if err := run(); err != nil {
log.Fatalf("fluid-control: %v", err)
}
}
func run() error {
var (
addr = flag.String("addr", "127.0.0.1:8081", "listen address")
store = flag.String("store", envOr("FLUID_STORE", "fluid.db"), "evidence store path")
iface = flag.String("interface", os.Getenv("FLUID_INTERFACE"), "interface identifier")
keyID = flag.String("key-id", envOr("FLUID_SIGNING_KEY_ID", "dev"), "signing key identifier")
keyFile = flag.String("key-file", os.Getenv("FLUID_SIGNING_KEY"), "base64 ed25519 private key file")
ephemeral = flag.Bool("ephemeral-key", false, "generate a throwaway signing key (development only)")
Add telemetry ingest, feedback collector, pressure API and insight CLI Completes FLUID-WP-0005. Normalization and redaction live on one path, shared by the in-process emitter and the ingest endpoint: two paths with two normalizations would eventually disagree, and the disagreement would surface as a pressure finding that is really a pipeline bug. Telemetry kind is inferred from event shape rather than defaulting to "request", since an error filed as a request understates the interface's failure rate. A malformed event in a batch does not discard the rest. Feedback is stored as evidence and creates no pressure and no hypothesis on its own, per API Standards 15, with the consumer recorded as the actor so their untrusted status stays visible in the audit trail. The feedback endpoint is the only consumer-reachable part of the control plane. The observation endpoints are not served at all when no pseudonymization salt is configured, rather than served with a generated one: a salt that changed per run would make the same consumer look new every time and every cohort count wrong. Adds an end-to-end test driving real traffic through the gateway and confirming it becomes a classified pressure record, with no raw consumer identity reaching the store. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014KmVxhJ35tCo7rE7UnLwWu Assistant: claude-code Assistant-Model: opus Assistant-Process: 1116572@bnt-lap001 Assistant-Session: 8ba9bb93-a72a-4883-b189-2499cce5c400
2026-09-04 03:19:06 +02:00
saltFile = flag.String("redaction-salt-file", os.Getenv("FLUID_REDACTION_SALT"),
"file holding the pseudonymization salt; required for the observation plane")
)
flag.Parse()
if *iface == "" {
return errors.New("--interface is required")
}
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
ev, err := evidence.OpenSQLite(ctx, *store)
if err != nil {
return err
}
defer ev.Close()
signer, err := loadSigner(*keyID, *keyFile, *ephemeral)
if err != nil {
return err
}
intents := intent.New(ev, contract.InterfaceID(*iface))
gate := policy.NewGate(policy.DefaultLimits())
pipeline, err := publish.New(publish.Options{
Gate: gate,
Signer: signer,
Store: ev,
Intents: intents,
})
if err != nil {
return err
}
Add telemetry ingest, feedback collector, pressure API and insight CLI Completes FLUID-WP-0005. Normalization and redaction live on one path, shared by the in-process emitter and the ingest endpoint: two paths with two normalizations would eventually disagree, and the disagreement would surface as a pressure finding that is really a pipeline bug. Telemetry kind is inferred from event shape rather than defaulting to "request", since an error filed as a request understates the interface's failure rate. A malformed event in a batch does not discard the rest. Feedback is stored as evidence and creates no pressure and no hypothesis on its own, per API Standards 15, with the consumer recorded as the actor so their untrusted status stays visible in the audit trail. The feedback endpoint is the only consumer-reachable part of the control plane. The observation endpoints are not served at all when no pseudonymization salt is configured, rather than served with a generated one: a salt that changed per run would make the same consumer look new every time and every cohort count wrong. Adds an end-to-end test driving real traffic through the gateway and confirming it becomes a classified pressure record, with no raw consumer identity reaching the store. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014KmVxhJ35tCo7rE7UnLwWu Assistant: claude-code Assistant-Model: opus Assistant-Process: 1116572@bnt-lap001 Assistant-Session: 8ba9bb93-a72a-4883-b189-2499cce5c400
2026-09-04 03:19:06 +02:00
// The observation plane is optional. Without a salt there is no safe way to
// pseudonymize consumer identities, so the endpoints that would record them
// are simply not served rather than served unsafely.
var pressureAPI *control.PressureAPI
if *saltFile != "" {
salt, err := os.ReadFile(*saltFile)
if err != nil {
return fmt.Errorf("read redaction salt: %w", err)
}
policy := observation.DefaultRedactionPolicy([]byte(trimSpace(string(salt))))
ingest, err := observation.NewIngest(ev, contract.InterfaceID(*iface), policy)
if err != nil {
return err
}
pressureAPI = control.NewPressureAPI(
observation.NewPressureRegistry(ev, contract.InterfaceID(*iface)), ingest)
} else {
log.Print("no redaction salt configured: telemetry, feedback and pressure endpoints are disabled")
}
Add science control APIs and the hypothesis, experiment and promote CLI Completes FLUID-WP-0006. The loop now runs end to end from the command line: two competing presentation hypotheses, an experiment that issues a routing policy rather than touching traffic, an amendment, a stop that returns traffic to the default, a confirmed outcome, a resolved competition, and a promotion the gate can refuse. Starting or stopping an experiment returns the routing policy for the operator to install rather than installing it. Blueprint 17 keeps the controller out of the traffic path, and installing from the handler would put it straight back in; emitting the document keeps the separation visible instead of implied. `fluid audit trace` now answers the section 25 questions from events rather than summary records, and names the rivals a hypothesis beat: an audit asking which hypotheses were considered is not answered by naming only the winner. Two fixes found by driving the CLI rather than only the tests. Go's flag package stops at the first positional, so ids given after flags silently swallowed them; ids are now taken before parsing. And there was no way to attach a revision to the hypothesis that produced it, which left `audit trace` unable to say why a revision existed -- `hypothesis attach` closes that. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014KmVxhJ35tCo7rE7UnLwWu Assistant: claude-code Assistant-Model: opus Assistant-Process: 1116572@bnt-lap001 Assistant-Session: 8ba9bb93-a72a-4883-b189-2499cce5c400
2026-09-04 06:44:47 +02:00
hypotheses := science.NewHypothesisStore(ev, contract.InterfaceID(*iface))
experiments := science.NewExperimentController(ev, hypotheses, contract.InterfaceID(*iface))
srv := &http.Server{
Add telemetry ingest, feedback collector, pressure API and insight CLI Completes FLUID-WP-0005. Normalization and redaction live on one path, shared by the in-process emitter and the ingest endpoint: two paths with two normalizations would eventually disagree, and the disagreement would surface as a pressure finding that is really a pipeline bug. Telemetry kind is inferred from event shape rather than defaulting to "request", since an error filed as a request understates the interface's failure rate. A malformed event in a batch does not discard the rest. Feedback is stored as evidence and creates no pressure and no hypothesis on its own, per API Standards 15, with the consumer recorded as the actor so their untrusted status stays visible in the audit trail. The feedback endpoint is the only consumer-reachable part of the control plane. The observation endpoints are not served at all when no pseudonymization salt is configured, rather than served with a generated one: a salt that changed per run would make the same consumer look new every time and every cohort count wrong. Adds an end-to-end test driving real traffic through the gateway and confirming it becomes a classified pressure record, with no raw consumer identity reaching the store. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014KmVxhJ35tCo7rE7UnLwWu Assistant: claude-code Assistant-Model: opus Assistant-Process: 1116572@bnt-lap001 Assistant-Session: 8ba9bb93-a72a-4883-b189-2499cce5c400
2026-09-04 03:19:06 +02:00
Addr: *addr,
Handler: control.NewServer(
control.NewRevisionAPI(ev, pipeline),
control.NewIntentAPI(intents, gate),
pressureAPI,
Add science control APIs and the hypothesis, experiment and promote CLI Completes FLUID-WP-0006. The loop now runs end to end from the command line: two competing presentation hypotheses, an experiment that issues a routing policy rather than touching traffic, an amendment, a stop that returns traffic to the default, a confirmed outcome, a resolved competition, and a promotion the gate can refuse. Starting or stopping an experiment returns the routing policy for the operator to install rather than installing it. Blueprint 17 keeps the controller out of the traffic path, and installing from the handler would put it straight back in; emitting the document keeps the separation visible instead of implied. `fluid audit trace` now answers the section 25 questions from events rather than summary records, and names the rivals a hypothesis beat: an audit asking which hypotheses were considered is not answered by naming only the winner. Two fixes found by driving the CLI rather than only the tests. Go's flag package stops at the first positional, so ids given after flags silently swallowed them; ids are now taken before parsing. And there was no way to attach a revision to the hypothesis that produced it, which left `audit trace` unable to say why a revision existed -- `hypothesis attach` closes that. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014KmVxhJ35tCo7rE7UnLwWu Assistant: claude-code Assistant-Model: opus Assistant-Process: 1116572@bnt-lap001 Assistant-Session: 8ba9bb93-a72a-4883-b189-2499cce5c400
2026-09-04 06:44:47 +02:00
control.NewScienceAPI(hypotheses, experiments),
Add telemetry ingest, feedback collector, pressure API and insight CLI Completes FLUID-WP-0005. Normalization and redaction live on one path, shared by the in-process emitter and the ingest endpoint: two paths with two normalizations would eventually disagree, and the disagreement would surface as a pressure finding that is really a pipeline bug. Telemetry kind is inferred from event shape rather than defaulting to "request", since an error filed as a request understates the interface's failure rate. A malformed event in a batch does not discard the rest. Feedback is stored as evidence and creates no pressure and no hypothesis on its own, per API Standards 15, with the consumer recorded as the actor so their untrusted status stays visible in the audit trail. The feedback endpoint is the only consumer-reachable part of the control plane. The observation endpoints are not served at all when no pseudonymization salt is configured, rather than served with a generated one: a salt that changed per run would make the same consumer look new every time and every cohort count wrong. Adds an end-to-end test driving real traffic through the gateway and confirming it becomes a classified pressure record, with no raw consumer identity reaching the store. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014KmVxhJ35tCo7rE7UnLwWu Assistant: claude-code Assistant-Model: opus Assistant-Process: 1116572@bnt-lap001 Assistant-Session: 8ba9bb93-a72a-4883-b189-2499cce5c400
2026-09-04 03:19:06 +02:00
).Routes(),
ReadHeaderTimeout: 10 * time.Second,
}
go func() {
<-ctx.Done()
shutdown, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
_ = srv.Shutdown(shutdown)
}()
log.Printf("control plane for %s listening on %s (store %s, signing key %s)",
*iface, *addr, *store, signer.KeyID())
log.Printf("public key: %s", base64.StdEncoding.EncodeToString(signer.PublicKey()))
if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
return err
}
return nil
}
// loadSigner resolves the signing key.
//
// An ephemeral key must be asked for explicitly. Generating one silently when
// none is configured would mean a deployment could sign revisions with a key
// nobody trusts and never notice until the router refused them.
func loadSigner(keyID, keyFile string, ephemeral bool) (*signing.Signer, error) {
if keyFile != "" {
raw, err := os.ReadFile(keyFile)
if err != nil {
return nil, fmt.Errorf("read signing key: %w", err)
}
decoded, err := base64.StdEncoding.DecodeString(trimSpace(string(raw)))
if err != nil {
return nil, fmt.Errorf("signing key is not valid base64: %w", err)
}
return signing.NewSigner(keyID, ed25519.PrivateKey(decoded))
}
if ephemeral {
signer, _, err := signing.GenerateKey(keyID)
if err != nil {
return nil, err
}
log.Print("WARNING: using an ephemeral signing key; revisions signed now " +
"will not verify after a restart")
return signer, nil
}
return nil, errors.New("no signing key: pass --key-file, or --ephemeral-key for development")
}
func trimSpace(s string) string {
start, end := 0, len(s)
for start < end && isSpace(s[start]) {
start++
}
for end > start && isSpace(s[end-1]) {
end--
}
return s[start:end]
}
func isSpace(b byte) bool {
return b == ' ' || b == '\t' || b == '\n' || b == '\r'
}
func envOr(key, fallback string) string {
if v := os.Getenv(key); v != "" {
return v
}
return fallback
}