diff --git a/SCOPE.md b/SCOPE.md index 223e364..aea0e45 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -76,9 +76,9 @@ read model. - Usage: internal Railiance ecosystem modeling, registry onboarding, and graph export work -The current adaptation wave for new Railiance repo families is -`RAIL-FAB-WP-0025`, which prepares the first compatibility-first Fabric support -for `rail-*`, `rapp-*`, and `reef-*`. +The current repo-family follow-on wave is `RAIL-FAB-WP-0026`, which projects +the first declaration-backed `rail-*`, `rapp-*`, and `reef-*` relations on top +of the compatibility-first groundwork from `RAIL-FAB-WP-0025`. --- diff --git a/WORK-RECORDS.md b/WORK-RECORDS.md index af3643d..761d61a 100644 --- a/WORK-RECORDS.md +++ b/WORK-RECORDS.md @@ -9,7 +9,11 @@ | Kind | ID | Status | Lane | Source | | --- | --- | --- | --- | --- | | workplan | RAIL-FAB-WP-0025 | finished | — | workplans/RAIL-FAB-WP-0025-rail-rapp-reef-fabric-adaptation.md | +| workplan | RAIL-FAB-WP-0026 | finished | — | workplans/RAIL-FAB-WP-0026-repo-family-relation-projection.md | | task | RAIL-FAB-WP-0025-T01 | done | — | workplans/RAIL-FAB-WP-0025-rail-rapp-reef-fabric-adaptation.md | | task | RAIL-FAB-WP-0025-T02 | done | — | workplans/RAIL-FAB-WP-0025-rail-rapp-reef-fabric-adaptation.md | | task | RAIL-FAB-WP-0025-T03 | done | — | workplans/RAIL-FAB-WP-0025-rail-rapp-reef-fabric-adaptation.md | | task | RAIL-FAB-WP-0025-T04 | done | — | workplans/RAIL-FAB-WP-0025-rail-rapp-reef-fabric-adaptation.md | +| task | RAIL-FAB-WP-0026-T01 | done | — | workplans/RAIL-FAB-WP-0026-repo-family-relation-projection.md | +| task | RAIL-FAB-WP-0026-T02 | done | — | workplans/RAIL-FAB-WP-0026-repo-family-relation-projection.md | +| task | RAIL-FAB-WP-0026-T03 | done | — | workplans/RAIL-FAB-WP-0026-repo-family-relation-projection.md | diff --git a/docs/rail-rapp-reef-adaptation.md b/docs/rail-rapp-reef-adaptation.md index e1369f8..a170d8f 100644 --- a/docs/rail-rapp-reef-adaptation.md +++ b/docs/rail-rapp-reef-adaptation.md @@ -1,6 +1,6 @@ # Rail Rapp Reef Adaptation -Date: 2026-07-25 +Date: 2026-07-26 ## Purpose @@ -79,6 +79,10 @@ This metadata is available through repository registration, manifest sync, and repository projections such as the graph explorer's synthesized repository nodes. +The registry now also retains the local checkout `path` used during onboarding. +That lets Fabric reopen the conventional repo-local `rail`, `rapp`, and `reef` +files when it needs to project first-wave relation edges. + ## First New Capability Type Fabric now reserves: @@ -95,23 +99,35 @@ Meaning: This is intentionally rail-focused. `rapp-*` and `reef-*` can start with service-level visibility before they need their own dedicated capability types. +## Relation Projection Path + +Fabric now projects the minimum first-wave repo-family relations from +repo-local files: + +- `governed_by` from `declarations/rail.yaml`, `declarations/rapp.yaml`, and + `declarations/reef.yaml` +- `supports_rail` from `declarations/rapp.yaml` +- `hosts_rail` from `bindings/rails.yaml` with a fallback to the reef + declaration +- `binds_rapp` from `bindings/rapps.yaml` with a fallback to the reef + declaration + +This keeps the relation path anchored in reviewed repo-local files rather than +guessing solely from registry metadata. + ## What Is Still Missing -The following are not done by this adaptation slice: +The following are still not done: -- export relations such as `supports_rail`, `hosts_rail`, `binds_rapp`, or - `governed_by` - validator rules for repo-family-specific declaration completeness -- graph queries that resolve placement or governance directly from the new - repo-family metadata +- richer graph queries that answer reef/workload placement beyond the first + projected repo edges - State Hub read-model fields that surface repo family directly in generated repo briefs or inventory views - -Those require a second wave after the first real `rail-*`, `rapp-*`, and -`reef-*` repos exist. +- deeper runtime placement semantics than the first repo-level topology ## Outcome Fabric can now move from "the new repo families exist only in architecture -docs" to "the registry and declaration guidance are ready for the first repo -instances" without overcommitting to a premature graph redesign. +docs" to "the registry can project the first repo-family topology from +repo-local evidence" without overcommitting to a premature graph redesign. diff --git a/docs/registry-onboarding.md b/docs/registry-onboarding.md index 3326434..3205ebd 100644 --- a/docs/registry-onboarding.md +++ b/docs/registry-onboarding.md @@ -51,7 +51,9 @@ repositories: sbom: bom.json ``` -`path` is the local checkout used for git metadata and default graph discovery. +`path` is the local checkout used for git metadata, default graph discovery, +and repo-family relation projection when the registry later needs to reopen +repo-local `rail`, `rapp`, or `reef` files. `declaration_paths` is optional; when omitted, the repo path is scanned for a `fabric/` directory. Relative paths are resolved from the manifest file. @@ -68,7 +70,7 @@ The following repo-family metadata fields are also optional: declared - `supported_rails`: array of supported rail ids for `rapp-*` repos - `substrate_kind`: grouped substrate category such as `server`, - `workstation-group`, or `cluster` + `server-group`, `workstation-group`, or `cluster` `sbom` or `sboms` may point to CycloneDX JSON/YAML files. When present, the command ingests them as queryable library inventory after repository @@ -81,6 +83,12 @@ no Fabric declarations yet, the command leaves the repo registered and reports a warning. This lets the registry represent known repos before every repo has published local evidence. +For `rail-*`, `rapp-*`, and `reef-*` repos, registration-only entries still +matter: the registry stores the checkout path and repo-family metadata, and the +graph export can use the conventional repo-local declaration files to project +first-wave `governed_by`, `supports_rail`, `hosts_rail`, and `binds_rapp` +edges even when the repo does not yet publish native Fabric declarations. + When declarations exist, the command validates them, builds a graph snapshot, and posts it to: diff --git a/railiance_fabric/canon.py b/railiance_fabric/canon.py index 69237ae..df6fe3e 100644 --- a/railiance_fabric/canon.py +++ b/railiance_fabric/canon.py @@ -114,6 +114,7 @@ EDGE_TYPE_CANON_MAP: dict[str, CanonEdgeMapping] = { "available_via": CanonEdgeMapping("exposes", "model/network", "partial"), "attributed_to_cost_center": CanonEdgeMapping("governed_by", "model/governance", "gap"), "attributed_to_profit_center": CanonEdgeMapping("governed_by", "model/governance", "gap"), + "binds_rapp": CanonEdgeMapping("deploys", "model/devsecops", "partial"), "binds": CanonEdgeMapping("depends_on", "model/landscape", "partial"), "builds_container": CanonEdgeMapping("built_from", "model/devsecops", "partial"), "cataloged_as": CanonEdgeMapping("evidenced_by", "model/observability", "partial"), @@ -135,12 +136,14 @@ EDGE_TYPE_CANON_MAP: dict[str, CanonEdgeMapping] = { "operated_by": CanonEdgeMapping("governed_by", "model/governance", "partial"), "owned_by": CanonEdgeMapping("governed_by", "model/governance", "partial"), "owns_deployment": CanonEdgeMapping("part_of", "model/devsecops", "partial", display_only=True), + "hosts_rail": CanonEdgeMapping("deploys", "model/devsecops", "partial"), "provides": CanonEdgeMapping("implements", "model/landscape", "partial"), "provides_utility_to": CanonEdgeMapping("depends_on", "model/landscape", "partial"), "resolves_to": CanonEdgeMapping("flows_to", "model/network", "partial"), "routes_to_port": CanonEdgeMapping("flows_to", "model/network", "partial"), "routes_to_service": CanonEdgeMapping("flows_to", "model/network", "partial"), "runs_on": CanonEdgeMapping("deploys", "model/devsecops", "partial"), + "supports_rail": CanonEdgeMapping("depends_on", "model/landscape", "partial"), "suggests_capability": CanonEdgeMapping("creates_task", "model/task", "partial"), "uses_config": CanonEdgeMapping("evidenced_by", "model/observability", "partial"), "uses_interface": CanonEdgeMapping("depends_on", "model/landscape", "partial"), diff --git a/railiance_fabric/cli.py b/railiance_fabric/cli.py index f14d4ae..b964e11 100644 --- a/railiance_fabric/cli.py +++ b/railiance_fabric/cli.py @@ -450,6 +450,7 @@ def _registry_sync(args: argparse.Namespace) -> int: { "slug": repo_slug, "name": args.name or repo_path.name, + "path": str(repo_path), "remote_url": args.remote_url or _git_value(repo_path, "config", "--get", "remote.origin.url"), "default_branch": args.default_branch, "state_hub_repo_id": args.state_hub_repo_id, @@ -457,6 +458,7 @@ def _registry_sync(args: argparse.Namespace) -> int: "ownership_repo": args.ownership_repo, "primary_rail": args.primary_rail, "supported_rails": args.supported_rail, + "declaration_paths": [str(path.resolve()) for path in args.paths], "substrate_kind": args.substrate_kind, }, ) @@ -834,7 +836,7 @@ def _scan_manifest_repo( repository = _registry_post_checked( registry_url, "/repositories", - _manifest_repository_payload(item, repo_path, slug), + _manifest_repository_payload(item, repo_path, slug, manifest_dir), ) stored = _registry_post_checked( registry_url, @@ -1234,7 +1236,7 @@ def _sync_manifest_repo(registry_url: str, manifest_dir: Path, item: object) -> repository = _registry_post_checked( registry_url, "/repositories", - _manifest_repository_payload(item, repo_path, slug), + _manifest_repository_payload(item, repo_path, slug, manifest_dir), ) result["repository"] = repository except RegistryRequestError as exc: @@ -1595,10 +1597,21 @@ def _manifest_optional_path(value: object, manifest_dir: Path) -> Path | None: return path if path.is_absolute() else (manifest_dir / path).resolve() -def _manifest_repository_payload(item: dict[str, Any], repo_path: Path | None, slug: str) -> dict[str, Any]: +def _manifest_repository_payload( + item: dict[str, Any], + repo_path: Path | None, + slug: str, + manifest_dir: Path | None = None, +) -> dict[str, Any]: + declaration_paths = ( + [str(path) for path in _manifest_paths(item.get("declaration_paths"), manifest_dir)] + if manifest_dir is not None + else [] + ) return { "slug": slug, "name": item.get("name") or (repo_path.name if repo_path else slug), + "path": str(repo_path) if repo_path is not None else None, "remote_url": item.get("remote_url") or _git_value(repo_path, "config", "--get", "remote.origin.url"), "default_branch": item.get("default_branch") or "main", "state_hub_repo_id": item.get("state_hub_repo_id"), @@ -1606,6 +1619,7 @@ def _manifest_repository_payload(item: dict[str, Any], repo_path: Path | None, s "ownership_repo": item.get("ownership_repo"), "primary_rail": item.get("primary_rail"), "supported_rails": item.get("supported_rails"), + "declaration_paths": declaration_paths, "substrate_kind": item.get("substrate_kind"), } diff --git a/railiance_fabric/graph_explorer.py b/railiance_fabric/graph_explorer.py index e702000..f1059af 100644 --- a/railiance_fabric/graph_explorer.py +++ b/railiance_fabric/graph_explorer.py @@ -61,12 +61,14 @@ _LAYER_COLORS = { _EDGE_STRENGTH = { "built_from": "medium", + "binds_rapp": "strong", "depends_on": "medium", "deploys": "strong", "evidenced_by": "medium", "exposes": "strong", "flows_to": "medium", "governed_by": "medium", + "hosts_rail": "strong", "implements": "medium", "observed_by": "medium", "part_of": "weak", @@ -86,6 +88,7 @@ _EDGE_STRENGTH = { "resolves_to": "medium", "runs_on": "strong", "owns_deployment": "weak", + "supports_rail": "medium", } @@ -325,6 +328,7 @@ def fabric_graph_explorer_payload( source_repo_slugs = { str(node.get("repo", "")).strip() for node in source_nodes + if str(node.get("kind", "")) != "Repository" if str(node.get("repo", "")).strip() } registered_repo_slugs = { @@ -347,6 +351,12 @@ def fabric_graph_explorer_payload( and str(node.get("id", "")) and str(node.get("repo", "")).strip() } + repository_source_references = { + str(node.get("repo", "")).strip(): _source_references(node) + for node in source_nodes + if str(node.get("kind", "")) == "Repository" + and str(node.get("repo", "")).strip() + } for slug in sorted(repo_slugs): repo = repository_index.get(slug, {}) has_snapshot = slug in snapshot_repo_slugs or slug in source_repo_slugs @@ -383,7 +393,7 @@ def fabric_graph_explorer_payload( "displayState": "show", "visibilitySource": "default", "visibilityReason": "default", - "sourceReferences": [], + "sourceReferences": repository_source_references.get(slug, []), "deepLinks": _repository_links(repo), }, "classes": "repository accepted" if has_snapshot else "repository candidate unresolved", diff --git a/railiance_fabric/registry.py b/railiance_fabric/registry.py index ec8d343..38123c8 100644 --- a/railiance_fabric/registry.py +++ b/railiance_fabric/registry.py @@ -16,7 +16,7 @@ from .financial import ( materialize_financial_graph_export, merge_financial_graph_exports, ) -from .loader import repo_root +from .loader import load_yaml, repo_root from .schema_validation import draft202012_validator RESET_CONFIRMATION_TOKEN = "RESET-RAILIANCE-FABRIC-GRAPH-DATA" @@ -42,6 +42,7 @@ class RegistryStore: create table if not exists repositories ( slug text primary key, name text not null, + path text, remote_url text, default_branch text, state_hub_repo_id text, @@ -49,6 +50,7 @@ class RegistryStore: ownership_repo text, primary_rail text, supported_rails_json text not null default '[]', + declaration_paths_json text not null default '[]', substrate_kind text, created_at text not null, updated_at text not null @@ -138,6 +140,7 @@ class RegistryStore: slug = _required_text(payload, "slug") now = _utc_now() name = str(payload.get("name") or slug) + path = _optional_text(payload, "path") remote_url = _optional_text(payload, "remote_url") default_branch = str(payload.get("default_branch") or "main") state_hub_repo_id = _optional_text(payload, "state_hub_repo_id") @@ -145,18 +148,20 @@ class RegistryStore: ownership_repo = _optional_text(payload, "ownership_repo") primary_rail = _optional_text(payload, "primary_rail") supported_rails = _optional_string_list(payload, "supported_rails") + declaration_paths = _optional_string_list(payload, "declaration_paths") substrate_kind = _optional_text(payload, "substrate_kind") with self._connect() as db: db.execute( """ insert into repositories ( - slug, name, remote_url, default_branch, state_hub_repo_id, + slug, name, path, remote_url, default_branch, state_hub_repo_id, repo_family, ownership_repo, primary_rail, supported_rails_json, - substrate_kind, created_at, updated_at + declaration_paths_json, substrate_kind, created_at, updated_at ) - values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) on conflict(slug) do update set name = excluded.name, + path = excluded.path, remote_url = excluded.remote_url, default_branch = excluded.default_branch, state_hub_repo_id = excluded.state_hub_repo_id, @@ -164,12 +169,14 @@ class RegistryStore: ownership_repo = excluded.ownership_repo, primary_rail = excluded.primary_rail, supported_rails_json = excluded.supported_rails_json, + declaration_paths_json = excluded.declaration_paths_json, substrate_kind = excluded.substrate_kind, updated_at = excluded.updated_at """, ( slug, name, + path, remote_url, default_branch, state_hub_repo_id, @@ -177,6 +184,7 @@ class RegistryStore: ownership_repo, primary_rail, json.dumps(supported_rails, sort_keys=True), + json.dumps(declaration_paths, sort_keys=True), substrate_kind, now, now, @@ -188,9 +196,10 @@ class RegistryStore: with self._connect() as db: rows = db.execute( """ - select slug, name, remote_url, default_branch, state_hub_repo_id, + select slug, name, path, remote_url, default_branch, state_hub_repo_id, repo_family, ownership_repo, primary_rail, - supported_rails_json, substrate_kind, created_at, updated_at + supported_rails_json, declaration_paths_json, substrate_kind, + created_at, updated_at from repositories order by slug """ @@ -201,9 +210,10 @@ class RegistryStore: with self._connect() as db: row = db.execute( """ - select slug, name, remote_url, default_branch, state_hub_repo_id, + select slug, name, path, remote_url, default_branch, state_hub_repo_id, repo_family, ownership_repo, primary_rail, - supported_rails_json, substrate_kind, created_at, updated_at + supported_rails_json, declaration_paths_json, substrate_kind, + created_at, updated_at from repositories where slug = ? """, @@ -474,6 +484,13 @@ class RegistryStore: for edge in graph.get("edges", []): if isinstance(edge, dict): edges.append(_edge_with_canon_metadata(edge)) + projected_nodes, projected_edges = _project_repository_family_graph(self.list_repositories()) + for node in projected_nodes: + node_id = str(node.get("id", "")) + if node_id: + nodes[node_id] = node + for edge in projected_edges: + edges.append(_edge_with_canon_metadata(edge)) return { "apiVersion": "railiance.fabric/v1alpha1", "kind": "FabricGraphExport", @@ -1538,11 +1555,17 @@ def _row_dict(row: sqlite3.Row) -> dict[str, Any]: def _repository_dict(row: sqlite3.Row) -> dict[str, Any]: data = _row_dict(row) raw_supported_rails = data.pop("supported_rails_json", "[]") + raw_declaration_paths = data.pop("declaration_paths_json", "[]") try: decoded = json.loads(raw_supported_rails or "[]") except json.JSONDecodeError: decoded = [] data["supported_rails"] = decoded if isinstance(decoded, list) else [] + try: + declaration_paths = json.loads(raw_declaration_paths or "[]") + except json.JSONDecodeError: + declaration_paths = [] + data["declaration_paths"] = declaration_paths if isinstance(declaration_paths, list) else [] return data @@ -1552,10 +1575,12 @@ def _ensure_repository_columns(db: sqlite3.Connection) -> None: for row in db.execute("pragma table_info(repositories)").fetchall() } additions = { + "path": "text", "repo_family": "text", "ownership_repo": "text", "primary_rail": "text", "supported_rails_json": "text not null default '[]'", + "declaration_paths_json": "text not null default '[]'", "substrate_kind": "text", } for name, ddl in additions.items(): @@ -1989,6 +2014,262 @@ def _optional_string_list(payload: dict[str, Any], key: str) -> list[str]: return result +def _project_repository_family_graph(repositories: list[dict[str, Any]]) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]: + nodes: dict[str, dict[str, Any]] = {} + edges: dict[tuple[str, str, str], dict[str, Any]] = {} + + for repository in repositories: + slug = str(repository.get("slug") or "").strip() + if not slug: + continue + projection = _repository_family_projection(repository) + nodes[_repository_graph_node_id(slug)] = _repository_projection_node(repository, projection) + + ownership_repo = projection.get("ownership_repo") + if isinstance(ownership_repo, str) and ownership_repo: + target_id = _repository_graph_node_id(ownership_repo) + nodes.setdefault(target_id, _placeholder_repository_node(ownership_repo)) + edge = _repository_projection_edge( + slug, + ownership_repo, + "governed_by", + projection, + ) + if edge is not None: + edges[_edge_key(edge)] = edge + + for rail_slug in _string_list(projection.get("supported_rails")): + target_id = _repository_graph_node_id(rail_slug) + nodes.setdefault(target_id, _placeholder_repository_node(rail_slug)) + edge = _repository_projection_edge(slug, rail_slug, "supports_rail", projection) + if edge is not None: + edges[_edge_key(edge)] = edge + + for rail_slug in _string_list(projection.get("hosted_rails")): + target_id = _repository_graph_node_id(rail_slug) + nodes.setdefault(target_id, _placeholder_repository_node(rail_slug)) + edge = _repository_projection_edge(slug, rail_slug, "hosts_rail", projection) + if edge is not None: + edges[_edge_key(edge)] = edge + + for rapp_slug in _string_list(projection.get("bound_rapps")): + target_id = _repository_graph_node_id(rapp_slug) + nodes.setdefault(target_id, _placeholder_repository_node(rapp_slug)) + edge = _repository_projection_edge(slug, rapp_slug, "binds_rapp", projection) + if edge is not None: + edges[_edge_key(edge)] = edge + + return [nodes[key] for key in sorted(nodes)], [edges[key] for key in sorted(edges)] + + +def _repository_family_projection(repository: dict[str, Any]) -> dict[str, Any]: + family = str(repository.get("repo_family") or "").strip() + repo_path = _repository_checkout_path(repository) + if repo_path is None or not repo_path.is_dir(): + return {} + + if family == "rail": + declaration_path = repo_path / "declarations" / "rail.yaml" + declaration = _load_yaml_mapping(declaration_path) + if declaration is None: + return {} + return { + "ownership_repo": _text_value(declaration, "ownership_repo"), + "source_links": [{"label": "Rail declaration", "path": str(declaration_path)}], + } + + if family == "rapp": + declaration_path = repo_path / "declarations" / "rapp.yaml" + declaration = _load_yaml_mapping(declaration_path) + if declaration is None: + return {} + supported_rails = _string_list(declaration.get("supported_rails")) + if not supported_rails: + primary_rail = _text_value(declaration, "primary_rail") + if primary_rail: + supported_rails = [primary_rail] + return { + "ownership_repo": _text_value(declaration, "ownership_repo"), + "supported_rails": supported_rails, + "source_links": [{"label": "Rapp declaration", "path": str(declaration_path)}], + } + + if family == "reef": + declaration_path = repo_path / "declarations" / "reef.yaml" + declaration = _load_yaml_mapping(declaration_path) + if declaration is None: + return {} + + rails_path = repo_path / "bindings" / "rails.yaml" + rails_binding = _load_yaml_mapping(rails_path) + hosted_rails = _relation_id_list( + rails_binding.get("hosted_rails") if isinstance(rails_binding, dict) else None, + key="rail_id", + ) + if not hosted_rails: + hosted_rails = _relation_id_list(declaration.get("hosted_rails"), key="rail_id") + if not hosted_rails: + primary_rail = _text_value(declaration, "primary_rail") + if primary_rail: + hosted_rails = [primary_rail] + + rapps_path = repo_path / "bindings" / "rapps.yaml" + rapps_binding = _load_yaml_mapping(rapps_path) + bound_rapps = _relation_id_list( + rapps_binding.get("bound_rapps") if isinstance(rapps_binding, dict) else None, + key="rapp_id", + ) + if not bound_rapps: + bound_rapps = _relation_id_list(declaration.get("bound_rapps"), key="rapp_id") + + source_links = [{"label": "Reef declaration", "path": str(declaration_path)}] + if rails_binding is not None: + source_links.append({"label": "Reef rail bindings", "path": str(rails_path)}) + if rapps_binding is not None: + source_links.append({"label": "Reef rapp bindings", "path": str(rapps_path)}) + return { + "ownership_repo": _text_value(declaration, "ownership_repo"), + "hosted_rails": hosted_rails, + "bound_rapps": bound_rapps, + "source_links": source_links, + } + + return {} + + +def _repository_checkout_path(repository: dict[str, Any]) -> Path | None: + raw_path = repository.get("path") + if not isinstance(raw_path, str) or not raw_path.strip(): + return None + return Path(raw_path).expanduser().resolve() + + +def _repository_graph_node_id(slug: str) -> str: + return f"repo:{slug}" + + +def _repository_projection_node(repository: dict[str, Any], projection: dict[str, Any]) -> dict[str, Any]: + slug = str(repository.get("slug") or "") + repo_id = _repository_graph_node_id(slug) + repo_name = str(repository.get("name") or slug) + source_links = projection.get("source_links") if isinstance(projection.get("source_links"), list) else [] + attributes: dict[str, Any] = { + "repo_family": str(repository.get("repo_family") or ""), + "ownership_repo": str(repository.get("ownership_repo") or ""), + "primary_rail": str(repository.get("primary_rail") or ""), + "supported_rails": _string_list(repository.get("supported_rails")), + "substrate_kind": str(repository.get("substrate_kind") or ""), + "path": str(repository.get("path") or ""), + "declaration_paths": _string_list(repository.get("declaration_paths")), + } + if source_links: + attributes["source_path"] = str(source_links[0].get("path") or "") + attributes["source_links"] = source_links + canon_mapping = node_canon_mapping("Repository") + return { + "id": repo_id, + "kind": "Repository", + "name": repo_name, + "repo": slug, + "domain": "railiance", + "lifecycle": "active", + "canon_category": canon_mapping.category, + "canon_anchor": canon_mapping.canon_anchor, + "mapping_fit": canon_mapping.fit, + "evidence_state": "declared", + "attributes": {key: value for key, value in attributes.items() if value not in ("", [], None)}, + } + + +def _placeholder_repository_node(slug: str) -> dict[str, Any]: + canon_mapping = node_canon_mapping("Repository") + return { + "id": _repository_graph_node_id(slug), + "kind": "Repository", + "name": slug, + "repo": slug, + "domain": "railiance", + "lifecycle": "registered-only", + "canon_category": canon_mapping.category, + "canon_anchor": canon_mapping.canon_anchor, + "mapping_fit": canon_mapping.fit, + "evidence_state": "inferred", + "attributes": {}, + } + + +def _repository_projection_edge( + source_slug: str, + target_slug: str, + edge_type: str, + projection: dict[str, Any], +) -> dict[str, Any] | None: + if not source_slug or not target_slug: + return None + source_links = projection.get("source_links") if isinstance(projection.get("source_links"), list) else [] + attributes: dict[str, Any] = {} + if source_links: + attributes["source_path"] = str(source_links[0].get("path") or "") + attributes["source_links"] = source_links + return { + "from": _repository_graph_node_id(source_slug), + "to": _repository_graph_node_id(target_slug), + "type": edge_type, + "evidence_state": "declared", + "attributes": attributes, + } + + +def _load_yaml_mapping(path: Path) -> dict[str, Any] | None: + if not path.is_file(): + return None + try: + data = load_yaml(path) + except Exception: + return None + return data if isinstance(data, dict) else None + + +def _text_value(payload: dict[str, Any], key: str) -> str: + value = payload.get(key) + return value.strip() if isinstance(value, str) else "" + + +def _string_list(value: object) -> list[str]: + if not isinstance(value, list): + return [] + result: list[str] = [] + seen: set[str] = set() + for item in value: + if not isinstance(item, str): + continue + cleaned = item.strip() + if not cleaned or cleaned in seen: + continue + seen.add(cleaned) + result.append(cleaned) + return result + + +def _relation_id_list(value: object, *, key: str) -> list[str]: + if isinstance(value, list): + result: list[str] = [] + seen: set[str] = set() + for item in value: + relation_id = "" + if isinstance(item, str): + relation_id = item.strip() + elif isinstance(item, dict): + raw = item.get(key) + relation_id = raw.strip() if isinstance(raw, str) else "" + if not relation_id or relation_id in seen: + continue + seen.add(relation_id) + result.append(relation_id) + return result + return [] + + def _utc_now() -> str: return datetime.now(timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z") diff --git a/registry/railiance-repos.yaml b/registry/railiance-repos.yaml index ad3ea8c..e6fe37b 100644 --- a/registry/railiance-repos.yaml +++ b/registry/railiance-repos.yaml @@ -85,6 +85,16 @@ repositories: supported_rails: - rail-kubernetes + - slug: reef-railiance + name: reef-railiance + path: /home/worsch/reef-railiance + remote_url: forgejo-remote:coulomb/reef-railiance.git + default_branch: main + repo_family: reef + ownership_repo: railiance-infra + primary_rail: rail-kubernetes + substrate_kind: server-group + - slug: flex-auth name: flex-auth path: /home/worsch/flex-auth diff --git a/tests/test_canon.py b/tests/test_canon.py index 25e46dc..013d1e0 100644 --- a/tests/test_canon.py +++ b/tests/test_canon.py @@ -56,6 +56,9 @@ def test_legacy_fabric_terms_map_without_becoming_display_edges() -> None: assert edge_canon_mapping("exposes").canonical_type == "exposes" assert edge_canon_mapping("provides").canonical_type == "implements" assert edge_canon_mapping("binds:exact").canonical_type == "depends_on" + assert edge_canon_mapping("supports_rail").canonical_type == "depends_on" + assert edge_canon_mapping("hosts_rail").canonical_type == "deploys" + assert edge_canon_mapping("binds_rapp").canonical_type == "deploys" assert edge_canon_mapping("resolves_to").canonical_type == "flows_to" assert edge_canon_mapping("declares").display_only is True diff --git a/tests/test_graph_explorer.py b/tests/test_graph_explorer.py index b227ad0..4ff70f4 100644 --- a/tests/test_graph_explorer.py +++ b/tests/test_graph_explorer.py @@ -159,6 +159,59 @@ def test_graph_explorer_collapses_discovered_repository_nodes() -> None: assert declares_package["data"]["displayOnly"] is False +def test_graph_explorer_surfaces_repository_source_references() -> None: + graph = { + "apiVersion": "railiance.fabric/v1alpha1", + "kind": "FabricGraphExport", + "generated_at": "2026-07-26T00:00:00Z", + "source": {"repo": "fixture-repo", "commit": "abc123"}, + "nodes": [ + { + "id": "repo:fixture-repo", + "kind": "Repository", + "name": "Fixture Repo", + "repo": "fixture-repo", + "domain": "testing", + "lifecycle": "active", + "attributes": { + "source_path": "/tmp/fixture-repo/declarations/rapp.yaml", + "source_links": [ + { + "label": "Rapp declaration", + "path": "/tmp/fixture-repo/declarations/rapp.yaml", + } + ], + }, + } + ], + "edges": [], + } + + payload = fabric_graph_explorer_payload( + graph, + [{"slug": "fixture-repo", "name": "Fixture Repo", "repo_family": "rapp"}], + {"fixture-repo"}, + ) + + repository_node = next( + element + for element in payload["elements"] + if element["data"].get("id") == "repo:fixture-repo" + and "source" not in element["data"] + ) + + assert repository_node["data"]["sourceReferences"] == [ + { + "label": "Declaration", + "path": "/tmp/fixture-repo/declarations/rapp.yaml", + }, + { + "label": "Rapp declaration", + "path": "/tmp/fixture-repo/declarations/rapp.yaml", + }, + ] + + def test_graph_explorer_presents_legacy_server_nodes_as_runtime_entities() -> None: graph = { "apiVersion": "railiance.fabric/v1alpha1", diff --git a/tests/test_registry.py b/tests/test_registry.py index 811b7c4..d50f7a7 100644 --- a/tests/test_registry.py +++ b/tests/test_registry.py @@ -102,6 +102,135 @@ def test_registry_accepts_snapshot_and_queries_graph(tmp_path: Path) -> None: assert "libraries" in library_xregistry_projection(store.list_libraries())["groups"] +def test_registry_projects_repo_family_relations_from_repo_local_declarations(tmp_path: Path) -> None: + store = RegistryStore(tmp_path / "registry.sqlite3") + store.init_schema() + store.upsert_repository({"slug": "railiance-cluster", "name": "Railiance Cluster"}) + store.upsert_repository({"slug": "railiance-infra", "name": "Railiance Infra"}) + store.upsert_repository({"slug": "railiance-platform", "name": "Railiance Platform"}) + + rail_repo = tmp_path / "rail-kubernetes" + (rail_repo / "declarations").mkdir(parents=True) + (rail_repo / "declarations" / "rail.yaml").write_text( + """ +kind: execution-rail +rail_id: rail-kubernetes +repo: rail-kubernetes +ownership_repo: railiance-cluster +""".lstrip(), + encoding="utf-8", + ) + rapp_repo = tmp_path / "rapp-openbao" + (rapp_repo / "declarations").mkdir(parents=True) + (rapp_repo / "declarations" / "rapp.yaml").write_text( + """ +kind: managed-workload-package +rapp_id: rapp-openbao +repo: rapp-openbao +ownership_repo: railiance-platform +primary_rail: rail-kubernetes +supported_rails: + - rail-kubernetes +""".lstrip(), + encoding="utf-8", + ) + reef_repo = tmp_path / "reef-railiance" + (reef_repo / "declarations").mkdir(parents=True) + (reef_repo / "bindings").mkdir(parents=True) + (reef_repo / "declarations" / "reef.yaml").write_text( + """ +kind: substrate-reef +reef_id: reef-railiance +repo: reef-railiance +ownership_repo: railiance-infra +primary_rail: rail-kubernetes +""".lstrip(), + encoding="utf-8", + ) + (reef_repo / "bindings" / "rails.yaml").write_text( + """ +reef_id: reef-railiance +hosted_rails: + - rail_id: rail-kubernetes + role: primary +""".lstrip(), + encoding="utf-8", + ) + (reef_repo / "bindings" / "rapps.yaml").write_text( + """ +reef_id: reef-railiance +bound_rapps: + - rapp_id: rapp-openbao + status: approved +""".lstrip(), + encoding="utf-8", + ) + + store.upsert_repository( + { + "slug": "rail-kubernetes", + "name": "rail-kubernetes", + "path": str(rail_repo), + "repo_family": "rail", + "ownership_repo": "railiance-cluster", + "declaration_paths": [str(rail_repo / "declarations" / "rail.yaml")], + } + ) + store.upsert_repository( + { + "slug": "rapp-openbao", + "name": "rapp-openbao", + "path": str(rapp_repo), + "repo_family": "rapp", + "ownership_repo": "railiance-platform", + "primary_rail": "rail-kubernetes", + "supported_rails": ["rail-kubernetes"], + "declaration_paths": [str(rapp_repo / "declarations" / "rapp.yaml")], + } + ) + store.upsert_repository( + { + "slug": "reef-railiance", + "name": "reef-railiance", + "path": str(reef_repo), + "repo_family": "reef", + "ownership_repo": "railiance-infra", + "primary_rail": "rail-kubernetes", + "declaration_paths": [str(reef_repo / "declarations" / "reef.yaml")], + } + ) + + combined = store.combined_graph() + nodes = {node["id"]: node for node in combined["nodes"] if node["kind"] == "Repository"} + edges = { + (edge["from"], edge["type"], edge["to"]): edge + for edge in combined["edges"] + } + + reef_record = store.get_repository("reef-railiance") + assert reef_record["path"] == str(reef_repo.resolve()) + assert reef_record["declaration_paths"] == [str((reef_repo / "declarations" / "reef.yaml").resolve())] + assert nodes["repo:reef-railiance"]["attributes"]["source_path"].endswith("declarations/reef.yaml") + assert ( + edges[("repo:rail-kubernetes", "governed_by", "repo:railiance-cluster")]["canonical_type"] + == "governed_by" + ) + assert ( + edges[("repo:rapp-openbao", "supports_rail", "repo:rail-kubernetes")]["canonical_type"] + == "depends_on" + ) + assert ( + edges[("repo:reef-railiance", "hosts_rail", "repo:rail-kubernetes")]["canonical_type"] + == "deploys" + ) + reef_binding = edges[("repo:reef-railiance", "binds_rapp", "repo:rapp-openbao")] + assert reef_binding["canonical_type"] == "deploys" + assert any( + str(link.get("path") or "").endswith("bindings/rapps.yaml") + for link in reef_binding["attributes"]["source_links"] + ) + + def test_registry_http_service_serves_queries(tmp_path: Path) -> None: store = RegistryStore(tmp_path / "registry.sqlite3") store.init_schema() diff --git a/workplans/RAIL-FAB-WP-0026-repo-family-relation-projection.md b/workplans/RAIL-FAB-WP-0026-repo-family-relation-projection.md new file mode 100644 index 0000000..8800f0f --- /dev/null +++ b/workplans/RAIL-FAB-WP-0026-repo-family-relation-projection.md @@ -0,0 +1,112 @@ +--- +id: RAIL-FAB-WP-0026 +type: workplan +title: "Project repo-family relations from repo-local declarations" +domain: financials +repo: railiance-fabric +status: finished +owner: codex +topic_slug: railiance +created: "2026-07-26" +updated: "2026-07-26" +state_hub_workstream_id: "13c95134-8a29-4b0a-a227-aa791f902517" +--- + +# RAIL-FAB-WP-0026 - Project repo-family relations from repo-local declarations + +## Goal + +Make the first `rail-*`, `rapp-*`, and `reef-*` repos visible in Fabric as a +minimal relation topology instead of only as isolated registered repositories. + +## Boundaries + +This workplan may: + +- persist repo checkout and declaration-location hints in the registry +- project repo-family relations from repo-local declaration files +- update graph-explorer and registry docs to surface the new relation path + +It must not: + +- redesign the broader Fabric graph model +- invent repo-family relations from registry metadata alone when source-backed + declaration files are available +- claim runtime placement or governance beyond what the repo-local files state + +## T01 - Persist local repo path context in repository records + +```task +id: RAIL-FAB-WP-0026-T01 +status: done +priority: high +state_hub_task_id: "e87f1771-718b-4b4f-9688-3673081dec67" +``` + +Store enough local checkout context in Fabric repository records so later graph +projection can reopen the relevant repo-local declarations. + +Acceptance: + +- repository records retain the checkout path when onboarding locally +- repository records retain declaration-path hints when they are provided +- the stored shape remains backward compatible for repos that only register by + slug and remote URL + +2026-07-26: Extended Fabric repository records to persist the local checkout +`path` and any provided `declaration_paths`, updated the registry schema +compatibly, and carried those fields through direct sync and manifest sync. + +## T02 - Project repo-family relations from repo-local files + +```task +id: RAIL-FAB-WP-0026-T02 +status: done +priority: high +state_hub_task_id: "d45a0de0-f7b3-46c0-9ee8-e306826863af" +``` + +Project the minimum repo-family relation topology from repo-local declaration +files instead of relying only on registry metadata. + +Acceptance: + +- Fabric can project `governed_by` from repo-local `rail`, `rapp`, and `reef` + declarations +- Fabric can project `supports_rail` from `rapp` declarations +- Fabric can project `hosts_rail` and `binds_rapp` from `reef` files +- the resulting relations appear in registry-backed exports + +2026-07-26: Added declaration-backed repo-family projection in the registry +combined graph. Fabric now projects `governed_by` from the first-wave `rail`, +`rapp`, and `reef` declarations, `supports_rail` from `rapp` declarations, and +`hosts_rail`/`binds_rapp` from the reef binding files with declaration +fallbacks. The graph explorer now preserves repository declaration references +for those projected repo nodes. + +## T03 - Verify the first-wave relation path end to end + +```task +id: RAIL-FAB-WP-0026-T03 +status: done +priority: medium +state_hub_task_id: "847cf1e9-8e29-4501-817b-b305e88302d9" +``` + +Verify that the current first-wave repos expose queryable repo-family relations +through the live Fabric registry/export path. + +Acceptance: + +- focused tests cover the new projection path +- the live registry export shows the first-wave ownership and rail-host/support + relations +- docs point operators at the declaration-backed projection path + +2026-07-26: Verified the projection with focused tests +(`tests/test_registry.py`, `tests/test_graph_explorer.py`, and +`tests/test_canon.py`) and reran them with local HTTP binding enabled. Restarted +the local registry, resynced `registry/railiance-repos.yaml`, and confirmed the +live export now emits `governed_by` for `rail-kubernetes`, `rapp-openbao`, and +`reef-railiance`, plus `supports_rail` for `rapp-openbao` and `hosts_rail` for +`reef-railiance`.