info-tech-canon/tests/test_maintenance.py

191 lines
7.5 KiB
Python
Raw Normal View History

from copy import deepcopy
import hashlib
import json
from pathlib import Path
import shutil
import tarfile
import pytest
import yaml
from info_tech_canon.cli import main
Declare concepts and review boundaries for the silent artifacts (T02) Twelve of the thirteen artifacts that declared nothing now declare what they define, together with the map-assigned concepts the Organization Model was missing and the Landscape seed concepts the extractor cannot see because they are listed rather than defined in prose. The ownership index grows from 164 entries to 750, undeclared prose definitions fall from 637 to 57, and there are no ownership conflicts. The 57 that remain are overlaps this round assigned to another owner: imports, each recorded in a boundary review. The kernel map declares nothing, deliberately, because it assigns concepts to owners rather than defining them. It is now the only silent artifact and the suite asserts that, so an artifact added without declarations fails. itc-org:Authority is declared, with Actor, Ownership, Membership, Role, Responsibility and Accountability, which SecurityCanon and the identity model already treat as organization-owned. The regression test that previously asserted the blind spot now asserts that the kernel map's assignment and the declaration agree. Profile is deliberately left undeclared. The kernel map assigns it to Core, the identity model owns it under accepted CUST-ADR-006, and Observability defines a runtime performance profile. Three senses need a decision, not a declaration, so it is recorded as open rather than forced. Eleven boundary reviews are added beside the artifacts they describe, in the shape the identity model has used since INFO-WP-0021. The largest finding is that Core and the Information Space model restate eight provenance concepts in nearly identical words; Core owns them and Information Space imports. Label, Drift, Summary and Attribute are resolved by disambiguation rather than transfer. make check passes with 50 tests, clean validation, no stale generated assets. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 3588@bnt-lap001 Assistant-Session: 24b80f66-e5a7-4e61-99fe-2d422e6d17da
2026-09-20 23:19:22 +02:00
from info_tech_canon.generation import concept_ownership
Measure the concept-declaration gap (INFO-WP-0027-T01) Adds maintenance.concept_candidates() and the concept-coverage CLI command, which measure concepts an artifact defines in prose against the concepts it declares. Extraction covers the bold form, the numbered-heading form that hid itc-org:Authority, and the concept-table form the kernel map uses; preserved source under assimilation, seeds and incoming is excluded. Candidates are review input, never ownership. Baseline over 31 live artifacts: 113 concepts declared against 690 defined, leaving 637 defined but undeclared, about 16 percent coverage. The workplan's 519 counted the bold form alone. Two corrections to the workplan's framing, applied there. Thirteen artifacts declare nothing rather than twelve: kernel/itc-core defines 57 concepts across two forms and declares none, and it is the artifact every other artifact imports from, so it goes first in T02. The gap also reaches further than obscure terms — Actor is undeclared in the organization model although SecurityCanon imports it from there by name against a pinned hash. Three tests cover the extractor, one asserting that Authority appears in the organization model's undeclared list, so the blind spot that produced finding F-1 now has a regression test. make check passes with 49 tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 3588@bnt-lap001 Assistant-Session: 24b80f66-e5a7-4e61-99fe-2d422e6d17da
2026-09-20 23:06:09 +02:00
from info_tech_canon.maintenance import (
check_generated,
concept_candidates,
export_emission_bundle,
source_evidence,
)
Report and enforce concept-declaration coverage (T03) validation-coverage gains a concept_declaration block: declared, defined in prose, undeclared, ratio, silent artifacts, and the extraction limit stated in words. The ratio sits slightly above one because seed-concept lists and YAML payload concepts are declared but not extractable, and saying so in the report is better than a number that looks complete. Two checks carry different weights. concept_declaration_missing is an error: an artifact that defines concepts and declares none, with the kernel map exempt by name because it assigns concepts rather than defining them. Zero today, so a new artifact added without declarations fails. concept_defined_without_owner is a warning over concepts no artifact declares; a name another artifact owns is an import rather than a gap, which keeps the warning from firing 57 times and training reviewers to ignore it. Three warnings today and each is real: the Organization Model defines eleven concepts nobody owns, CARING four including Effective Access and Declared Access, and the Capability Model two. They are carried as T07 rather than declared in passing, because declaring without a boundary review is the mistake this workplan exists to fix. make check passes with 53 tests, clean validation and three warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 3588@bnt-lap001 Assistant-Session: 24b80f66-e5a7-4e61-99fe-2d422e6d17da
2026-09-20 23:24:27 +02:00
from info_tech_canon.contracts import coverage
from info_tech_canon.service import (
DEFAULT_INFOSPACE_ROOT,
concept_declaration_checks,
load_context,
validate_canon,
)
@pytest.fixture
def corpus(tmp_path):
root = tmp_path / "infospace"
shutil.copytree(DEFAULT_INFOSPACE_ROOT, root)
return root
def test_emission_cli_checks_versions_duplicates_and_bad_types(tmp_path, capsys):
source = DEFAULT_INFOSPACE_ROOT / "standards/emission-cadence/examples/qonto-assistant.yaml"
valid = yaml.safe_load(source.read_text())
path = tmp_path / "declaration.yaml"
path.write_text(yaml.safe_dump(valid))
assert main(["emission-review", str(path)]) == 0
assert json.loads(capsys.readouterr().out)["operational_truth_assessed"] is False
for change in ("version", "duplicate", "bad_type"):
data = deepcopy(valid)
if change == "version":
data["schema_version"] = "99.0"
elif change == "duplicate":
data["sources"].append(deepcopy(data["sources"][0]))
else:
data["sources"][0]["source_id"] = ["unhashable"]
path.write_text(yaml.safe_dump(data))
assert main(["emission-review", str(path)]) == 1
payload = json.loads(capsys.readouterr().out)
assert payload["errors"]
if change == "duplicate":
assert payload["errors"][0]["code"] == "duplicate_emission_cadence_source_id"
def test_freshness_detects_missing_and_stale_without_repair(corpus):
from info_tech_canon import generation
context = load_context(corpus)
for render in (generation.generate_indexes, generation.generate_tree, generation.generate_agent_briefs):
render(context)
assert check_generated(context)["ok"]
stale = corpus / "views/by-concept.md"
stale.write_text("stale\n")
missing = corpus / "agent/retrieval-index.json"
missing.unlink()
result = check_generated(context)
assert not result["ok"]
assert "views/by-concept.md" in result["stale"]
assert "agent/retrieval-index.json" in result["stale"]
assert stale.read_text() == "stale\n"
assert not missing.exists()
def test_bundle_is_reproducible_and_refuses_corruption(tmp_path):
first = export_emission_bundle(DEFAULT_INFOSPACE_ROOT, tmp_path)
second = export_emission_bundle(DEFAULT_INFOSPACE_ROOT, tmp_path)
assert first == second
path = Path(first["path"])
assert hashlib.sha256(path.read_bytes()).hexdigest() == first["sha256"]
with tarfile.open(path) as archive:
for name, digest in first["manifest"]["files"].items():
assert hashlib.sha256(archive.extractfile(name).read()).hexdigest() == digest
path.write_bytes(b"corrupted")
with pytest.raises(ValueError, match="Refusing"):
export_emission_bundle(DEFAULT_INFOSPACE_ROOT, tmp_path)
def test_mapping_schema_and_explicit_ownership_are_enforced(corpus):
from info_tech_canon.contracts import bound_artifact_errors
mapping = corpus / "mappings/capability-anchors.yaml"
data = yaml.safe_load(mapping.read_text())
del data["target"]
mapping.write_text(yaml.safe_dump(data))
assert bound_artifact_errors(load_context(corpus))[0]["code"] == "schema_violation"
pattern = corpus / "patterns/AgenticDrivesFunctional.md"
text = pattern.read_text()
# Reuse another artifact's title as an owned concept to create a real conflict.
owner = next(a.title for a in load_context(corpus).infospace.artifacts if a.kind == "kernel")
text = text.replace("owned_concepts:", f"owned_concepts:\n - {owner}")
pattern.write_text(text)
assert any(e["code"] == "concept_ownership_conflict" for e in validate_canon(corpus)["errors"])
def test_evidence_digest_tracks_content_but_excludes_reports(corpus):
before = source_evidence(corpus)
(corpus / "validation/latest.json").write_text("{}")
assert source_evidence(corpus)["corpus_sha256"] == before["corpus_sha256"]
(corpus / "mappings/README.md").write_text("changed")
assert source_evidence(corpus)["corpus_sha256"] != before["corpus_sha256"]
assert before["generated_at"]
def test_explicit_root_controls_capability_catalog(corpus, tmp_path):
from info_tech_canon.service import review_capability_record, CanonServiceError
(corpus / "models/capability/capabilities.yaml").unlink()
record = tmp_path / "record.json"
record.write_text('{"record_id":"example", "requires":[], "provisions":[]}')
with pytest.raises(CanonServiceError, match="catalog"):
review_capability_record(record, corpus)
def test_missing_mapping_schema_is_a_validation_finding(corpus):
(corpus / "schemas/mapping.schema.yaml").unlink()
payload = validate_canon(corpus)
assert not payload["ok"]
assert any(e["code"] == "mapping_schema_unreadable" for e in payload["errors"])
Measure the concept-declaration gap (INFO-WP-0027-T01) Adds maintenance.concept_candidates() and the concept-coverage CLI command, which measure concepts an artifact defines in prose against the concepts it declares. Extraction covers the bold form, the numbered-heading form that hid itc-org:Authority, and the concept-table form the kernel map uses; preserved source under assimilation, seeds and incoming is excluded. Candidates are review input, never ownership. Baseline over 31 live artifacts: 113 concepts declared against 690 defined, leaving 637 defined but undeclared, about 16 percent coverage. The workplan's 519 counted the bold form alone. Two corrections to the workplan's framing, applied there. Thirteen artifacts declare nothing rather than twelve: kernel/itc-core defines 57 concepts across two forms and declares none, and it is the artifact every other artifact imports from, so it goes first in T02. The gap also reaches further than obscure terms — Actor is undeclared in the organization model although SecurityCanon imports it from there by name against a pinned hash. Three tests cover the extractor, one asserting that Authority appears in the organization model's undeclared list, so the blind spot that produced finding F-1 now has a regression test. make check passes with 49 tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 3588@bnt-lap001 Assistant-Session: 24b80f66-e5a7-4e61-99fe-2d422e6d17da
2026-09-20 23:06:09 +02:00
Declare concepts and review boundaries for the silent artifacts (T02) Twelve of the thirteen artifacts that declared nothing now declare what they define, together with the map-assigned concepts the Organization Model was missing and the Landscape seed concepts the extractor cannot see because they are listed rather than defined in prose. The ownership index grows from 164 entries to 750, undeclared prose definitions fall from 637 to 57, and there are no ownership conflicts. The 57 that remain are overlaps this round assigned to another owner: imports, each recorded in a boundary review. The kernel map declares nothing, deliberately, because it assigns concepts to owners rather than defining them. It is now the only silent artifact and the suite asserts that, so an artifact added without declarations fails. itc-org:Authority is declared, with Actor, Ownership, Membership, Role, Responsibility and Accountability, which SecurityCanon and the identity model already treat as organization-owned. The regression test that previously asserted the blind spot now asserts that the kernel map's assignment and the declaration agree. Profile is deliberately left undeclared. The kernel map assigns it to Core, the identity model owns it under accepted CUST-ADR-006, and Observability defines a runtime performance profile. Three senses need a decision, not a declaration, so it is recorded as open rather than forced. Eleven boundary reviews are added beside the artifacts they describe, in the shape the identity model has used since INFO-WP-0021. The largest finding is that Core and the Information Space model restate eight provenance concepts in nearly identical words; Core owns them and Information Space imports. Label, Drift, Summary and Attribute are resolved by disambiguation rather than transfer. make check passes with 50 tests, clean validation, no stale generated assets. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 3588@bnt-lap001 Assistant-Session: 24b80f66-e5a7-4e61-99fe-2d422e6d17da
2026-09-20 23:19:22 +02:00
def test_authority_is_declared_where_the_kernel_map_assigns_it():
Measure the concept-declaration gap (INFO-WP-0027-T01) Adds maintenance.concept_candidates() and the concept-coverage CLI command, which measure concepts an artifact defines in prose against the concepts it declares. Extraction covers the bold form, the numbered-heading form that hid itc-org:Authority, and the concept-table form the kernel map uses; preserved source under assimilation, seeds and incoming is excluded. Candidates are review input, never ownership. Baseline over 31 live artifacts: 113 concepts declared against 690 defined, leaving 637 defined but undeclared, about 16 percent coverage. The workplan's 519 counted the bold form alone. Two corrections to the workplan's framing, applied there. Thirteen artifacts declare nothing rather than twelve: kernel/itc-core defines 57 concepts across two forms and declares none, and it is the artifact every other artifact imports from, so it goes first in T02. The gap also reaches further than obscure terms — Actor is undeclared in the organization model although SecurityCanon imports it from there by name against a pinned hash. Three tests cover the extractor, one asserting that Authority appears in the organization model's undeclared list, so the blind spot that produced finding F-1 now has a regression test. make check passes with 49 tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 3588@bnt-lap001 Assistant-Session: 24b80f66-e5a7-4e61-99fe-2d422e6d17da
2026-09-20 23:06:09 +02:00
"""The gap that let itc-org:Authority pass the SecurityCanon conflict check."""
Declare concepts and review boundaries for the silent artifacts (T02) Twelve of the thirteen artifacts that declared nothing now declare what they define, together with the map-assigned concepts the Organization Model was missing and the Landscape seed concepts the extractor cannot see because they are listed rather than defined in prose. The ownership index grows from 164 entries to 750, undeclared prose definitions fall from 637 to 57, and there are no ownership conflicts. The 57 that remain are overlaps this round assigned to another owner: imports, each recorded in a boundary review. The kernel map declares nothing, deliberately, because it assigns concepts to owners rather than defining them. It is now the only silent artifact and the suite asserts that, so an artifact added without declarations fails. itc-org:Authority is declared, with Actor, Ownership, Membership, Role, Responsibility and Accountability, which SecurityCanon and the identity model already treat as organization-owned. The regression test that previously asserted the blind spot now asserts that the kernel map's assignment and the declaration agree. Profile is deliberately left undeclared. The kernel map assigns it to Core, the identity model owns it under accepted CUST-ADR-006, and Observability defines a runtime performance profile. Three senses need a decision, not a declaration, so it is recorded as open rather than forced. Eleven boundary reviews are added beside the artifacts they describe, in the shape the identity model has used since INFO-WP-0021. The largest finding is that Core and the Information Space model restate eight provenance concepts in nearly identical words; Core owns them and Information Space imports. Label, Drift, Summary and Attribute are resolved by disambiguation rather than transfer. make check passes with 50 tests, clean validation, no stale generated assets. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 3588@bnt-lap001 Assistant-Session: 24b80f66-e5a7-4e61-99fe-2d422e6d17da
2026-09-20 23:19:22 +02:00
ownership = concept_ownership(load_context())
owners = {item["concept"]: item["owner"] for item in ownership["concepts"]}
assert owners["Authority"] == "model/organization"
assert owners["Actor"] == "model/organization"
assert not ownership["ownership_conflicts"]
def test_only_the_kernel_map_declares_no_concepts():
"""It assigns concepts to owners rather than defining them (T02)."""
Measure the concept-declaration gap (INFO-WP-0027-T01) Adds maintenance.concept_candidates() and the concept-coverage CLI command, which measure concepts an artifact defines in prose against the concepts it declares. Extraction covers the bold form, the numbered-heading form that hid itc-org:Authority, and the concept-table form the kernel map uses; preserved source under assimilation, seeds and incoming is excluded. Candidates are review input, never ownership. Baseline over 31 live artifacts: 113 concepts declared against 690 defined, leaving 637 defined but undeclared, about 16 percent coverage. The workplan's 519 counted the bold form alone. Two corrections to the workplan's framing, applied there. Thirteen artifacts declare nothing rather than twelve: kernel/itc-core defines 57 concepts across two forms and declares none, and it is the artifact every other artifact imports from, so it goes first in T02. The gap also reaches further than obscure terms — Actor is undeclared in the organization model although SecurityCanon imports it from there by name against a pinned hash. Three tests cover the extractor, one asserting that Authority appears in the organization model's undeclared list, so the blind spot that produced finding F-1 now has a regression test. make check passes with 49 tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 3588@bnt-lap001 Assistant-Session: 24b80f66-e5a7-4e61-99fe-2d422e6d17da
2026-09-20 23:06:09 +02:00
report = concept_candidates(load_context())
Declare concepts and review boundaries for the silent artifacts (T02) Twelve of the thirteen artifacts that declared nothing now declare what they define, together with the map-assigned concepts the Organization Model was missing and the Landscape seed concepts the extractor cannot see because they are listed rather than defined in prose. The ownership index grows from 164 entries to 750, undeclared prose definitions fall from 637 to 57, and there are no ownership conflicts. The 57 that remain are overlaps this round assigned to another owner: imports, each recorded in a boundary review. The kernel map declares nothing, deliberately, because it assigns concepts to owners rather than defining them. It is now the only silent artifact and the suite asserts that, so an artifact added without declarations fails. itc-org:Authority is declared, with Actor, Ownership, Membership, Role, Responsibility and Accountability, which SecurityCanon and the identity model already treat as organization-owned. The regression test that previously asserted the blind spot now asserts that the kernel map's assignment and the declaration agree. Profile is deliberately left undeclared. The kernel map assigns it to Core, the identity model owns it under accepted CUST-ADR-006, and Observability defines a runtime performance profile. Three senses need a decision, not a declaration, so it is recorded as open rather than forced. Eleven boundary reviews are added beside the artifacts they describe, in the shape the identity model has used since INFO-WP-0021. The largest finding is that Core and the Information Space model restate eight provenance concepts in nearly identical words; Core owns them and Information Space imports. Label, Drift, Summary and Attribute are resolved by disambiguation rather than transfer. make check passes with 50 tests, clean validation, no stale generated assets. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 3588@bnt-lap001 Assistant-Session: 24b80f66-e5a7-4e61-99fe-2d422e6d17da
2026-09-20 23:19:22 +02:00
assert report["silent_artifacts"] == ["kernel/itc-kernel-map"]
assert report["undeclared_total"] < report["candidate_total"] / 2
Measure the concept-declaration gap (INFO-WP-0027-T01) Adds maintenance.concept_candidates() and the concept-coverage CLI command, which measure concepts an artifact defines in prose against the concepts it declares. Extraction covers the bold form, the numbered-heading form that hid itc-org:Authority, and the concept-table form the kernel map uses; preserved source under assimilation, seeds and incoming is excluded. Candidates are review input, never ownership. Baseline over 31 live artifacts: 113 concepts declared against 690 defined, leaving 637 defined but undeclared, about 16 percent coverage. The workplan's 519 counted the bold form alone. Two corrections to the workplan's framing, applied there. Thirteen artifacts declare nothing rather than twelve: kernel/itc-core defines 57 concepts across two forms and declares none, and it is the artifact every other artifact imports from, so it goes first in T02. The gap also reaches further than obscure terms — Actor is undeclared in the organization model although SecurityCanon imports it from there by name against a pinned hash. Three tests cover the extractor, one asserting that Authority appears in the organization model's undeclared list, so the blind spot that produced finding F-1 now has a regression test. make check passes with 49 tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 3588@bnt-lap001 Assistant-Session: 24b80f66-e5a7-4e61-99fe-2d422e6d17da
2026-09-20 23:06:09 +02:00
def test_concept_candidates_ignore_preserved_source():
report = concept_candidates(load_context())
paths = [item["path"] for item in report["artifacts"]]
assert paths
assert not [path for path in paths if path.startswith(("assimilation/", "seeds/"))]
def test_concept_coverage_cli_reports_one_artifact(capsys):
assert main(["concept-coverage", "--artifact", "model/identity"]) == 0
payload = json.loads(capsys.readouterr().out)
assert payload["ok"] is True
assert [item["artifact"] for item in payload["artifacts"]] == ["model/identity"]
Report and enforce concept-declaration coverage (T03) validation-coverage gains a concept_declaration block: declared, defined in prose, undeclared, ratio, silent artifacts, and the extraction limit stated in words. The ratio sits slightly above one because seed-concept lists and YAML payload concepts are declared but not extractable, and saying so in the report is better than a number that looks complete. Two checks carry different weights. concept_declaration_missing is an error: an artifact that defines concepts and declares none, with the kernel map exempt by name because it assigns concepts rather than defining them. Zero today, so a new artifact added without declarations fails. concept_defined_without_owner is a warning over concepts no artifact declares; a name another artifact owns is an import rather than a gap, which keeps the warning from firing 57 times and training reviewers to ignore it. Three warnings today and each is real: the Organization Model defines eleven concepts nobody owns, CARING four including Effective Access and Declared Access, and the Capability Model two. They are carried as T07 rather than declared in passing, because declaring without a boundary review is the mistake this workplan exists to fix. make check passes with 53 tests, clean validation and three warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 3588@bnt-lap001 Assistant-Session: 24b80f66-e5a7-4e61-99fe-2d422e6d17da
2026-09-20 23:24:27 +02:00
def test_declaration_checks_warn_on_unowned_and_error_on_silence(tmp_path):
context = load_context()
ownership = concept_ownership(context)
result = concept_declaration_checks(context, ownership)
assert not result["errors"]
warned = {item["artifact_id"] for item in result["warnings"]}
assert "model/organization" in warned
assert all(item["code"] == "concept_defined_without_owner" for item in result["warnings"])
def test_declaration_error_fires_when_an_artifact_declares_nothing(corpus):
target = corpus / "models/security/InfoTechCanonSecurityModel.md"
text = target.read_text(encoding="utf-8")
target.write_text(text.split("---\n", 2)[2], encoding="utf-8")
context = load_context(corpus)
result = concept_declaration_checks(context, concept_ownership(context))
assert [item["artifact_id"] for item in result["errors"]] == ["model/security"]
def test_validation_coverage_reports_the_declaration_ratio():
report = coverage(load_context())["concept_declaration"]
assert report["silent_artifacts"] == ["kernel/itc-kernel-map"]
assert report["undeclared"] < report["declared"]
assert report["limit"]