railiance-fabric/SuggestedRedesign.md

# Suggested Redesign: Forgejo-Backed Fabric Repository Model

## Purpose

`railiance-fabric` currently operates mainly on local working copies. Repository
manifests contain checkout paths, scanners read those directories, and the
registry stores the resulting snapshots. This is useful for development, but it
makes the local filesystem an accidental source of truth.

Fabric should instead model the repositories as they exist in Forgejo. Local
checkouts should be optional inputs for development and recovery. Forgejo owns
repository identity and source history; Fabric owns the accepted interpretation
of that evidence as an ecosystem and financial graph.

## Proposed flow

```text
Forgejo repositories
        |
        | API reconciliation, webhooks, and scheduled sync
        v
Forgejo repository inventory and source evidence index
        |
        | checkout-free scans at commit SHAs
        v
Validated discovery and graph snapshots
        |
        v
Accepted financial Fabric graph and State Hub projection
```

The registry should be able to discover repository metadata and read selected
files from Forgejo without cloning repositories. A local checkout adapter may
remain, but it should implement the same source interface as the Forgejo
adapter rather than define the model.

## Source-of-truth boundary

Forgejo is authoritative for:

- repository identity, namespace, and canonical URL;
- visibility, archival state, topics, and repository metadata;
- default branch, branches, tags, and commit history;
- commit contents and file paths.

Fabric is authoritative for:

- canonical repository identity across Forgejo and State Hub;
- repository family and ownership classification;
- accountability, containment, and financial Fabric membership;
- parsed declarations and discovered entities;
- validation, review, and acceptance state;
- graph relations and State Hub exports.

State Hub remains a read model for coordination and planning. It should link to
Fabric and Forgejo evidence rather than author repository topology.

## Data retention policy

Fabric must not store complete repository source content. It should retain only
the minimum evidence needed to reproduce and review the graph projection:

- Forgejo repository and commit identifiers;
- branch or tag used for a scan;
- relevant file paths;
- content hashes for relevant files;
- selected parsed declarations and discovery results;
- validation results and diagnostics;
- immutable graph and discovery snapshots;
- source URLs and commit-scoped provenance;
- optional references to larger documents stored in Forgejo or object storage.

Complete source archives, unrelated files, and full working trees remain in
Forgejo. If a large artifact such as an SBOM or API document must be retained,
Fabric may store it in object storage and keep only its URI, digest, media type,
and provenance in the database.

## Registry model

The current `repositories` and commit-scoped snapshot tables are a useful
prototype. The hosted registry should evolve them into a normalized store with
the following logical entities:

| Entity | Responsibility |
|---|---|
| Forgejo instance | Instance URL and connection identity, without secrets in graph data |
| Repository | Canonical Forgejo identity, URLs, status, branch metadata, State Hub link, and Fabric classification |
| Ref | Branch or tag and its current commit SHA |
| Commit | Commit identity, author metadata where needed, and parent relationship |
| Source file index | Relevant path, commit SHA, content digest, size, and retrieval provenance; no complete source by default |
| Repository scan | Profile, commit, scan time, status, tool version, and input evidence |
| Validation result | Errors, warnings, schema versions, and unresolved references |
| Discovery snapshot | Immutable checkout-free discovery result |
| Graph snapshot | Immutable accepted or candidate graph projection |
| Artifact | External or object-storage URI, digest, type, and target graph entity |

PostgreSQL is the appropriate production backing store for these indexed and
relational records. SQLite can remain useful for local development and export
rehearsal. The database should be treated as a rebuildable index over Forgejo
and accepted evidence, with snapshot history retained for auditability.

## Synchronization

Synchronization should be incremental and commit-scoped:

1. Reconcile Forgejo repositories into the registry.
2. Record branch and tag changes.
3. Accept Forgejo push, repository, branch, and tag webhooks where available.
4. Queue a scan for each affected commit and repository profile.
5. Fetch only the relevant tree entries and file blobs through the Forgejo API.
6. Compare file digests with the previous scan and skip unchanged inputs.
7. Parse and validate the selected evidence.
8. Store a discovery snapshot and candidate graph snapshot.
9. Review or promote the candidate according to the acceptance policy.
10. Rebuild the combined financial graph from accepted snapshots.

Webhooks provide low latency, while a scheduled reconciliation job detects
missed events, deleted repositories, force pushes, and stale branches. A failed
Forgejo request should produce visible synchronization state and retry metadata;
it should not silently remove the last accepted graph.

## Checkout-free source adapter

The scanner should consume a small source-provider interface with operations
such as:

- list repository tree entries at a commit;
- read a selected file at a commit;
- resolve a branch or tag to a commit;
- return repository and commit provenance.

The Forgejo implementation calls the Forgejo API. The local implementation
reads a checkout and uses Git. Both providers should produce the same scan
inputs and deterministic snapshot format. This permits local testing without
making local paths part of the production identity model.

## Financial graph integration

Every canonical Forgejo repository should be eligible to produce a `Repository`
node in the `railiance.fabric/v1alpha2` financial export. The node should carry
containment, ownership resolution, evidence state, and the Forgejo commit or
repository references used to establish it.

Forgejo metadata alone can establish that a repository exists. It cannot by
itself establish who pays for its infrastructure, which fabric contains it, or
which cross-boundary utility it provides. Those facts must continue to come
from accountability-root evidence, deployment evidence, reviewed registry
metadata, and accepted discovery snapshots.

Missing or conflicting ownership and family data should be emitted as
`unresolved` review gaps. It should not be inferred solely from repository
names or namespaces without recording that inference and its confidence.

## API and operational changes

The registry should add or evolve endpoints for:

```text
POST /sources/forgejo/reconcile
GET  /sources/forgejo/status
GET  /repositories/{repo_slug}/refs
GET  /repositories/{repo_slug}/commits/{sha}
POST /repositories/{repo_slug}/scans
GET  /repositories/{repo_slug}/scans/latest
```

Mutation endpoints require authentication and authorization when the registry
is hosted. Read endpoints should expose commit-scoped provenance, validation
state, and accepted snapshot identity so consumers can distinguish current
Forgejo state from accepted Fabric truth.

The hosted authority should retain accepted snapshots and deterministic
snapshot-set provenance. Backups need to cover the PostgreSQL registry and any
object-storage artifacts, while Forgejo remains the source for repository
content.

## Suggested implementation sequence

1. Define canonical Forgejo repository identity and alias rules.
2. Add Forgejo instance, repository, ref, commit, source-file-index, and scan
   state fields to the registry model.
3. Implement a Forgejo API source provider and recorded API fixtures.
4. Refactor scanning to consume the source-provider interface.
5. Scan one selected branch without a local checkout and store a deterministic
   discovery snapshot.
6. Project registered repositories into the financial export with provenance
   and unresolved-gap reporting.
7. Add webhook ingestion plus scheduled reconciliation and retry handling.
8. Migrate the hosted registry to PostgreSQL under `RAIL-FAB-WP-0028`.
9. Switch State Hub consumers to the hosted Fabric export endpoint.

The first milestone is complete when a repository can be registered, scanned at
a Forgejo commit, validated, and projected into the financial graph without
the repository being present on the Fabric host and without Fabric storing its
complete source content.