markitect-main/roadmap/schema-of-schemas/SCHEMA_NAMING_SPEC.md

409 lines
10 KiB
Markdown
Raw Normal View History

feat: implement schema filename validation (Phase 1 complete) Implements filename convention enforcement for schema files as part of the schema-of-schemas implementation. All schemas must now follow the naming pattern: {domain}-schema-v{major}.{minor}.md ## Phase 1 Deliverables ### Schema Naming Module **File:** `markitect/schema_naming.py` (380 lines) **Functions:** - `validate_schema_filename()` - Validate filename against pattern - `suggest_schema_filename()` - Generate valid filename from domain/version - `extract_schema_metadata()` - Extract domain and version from filename - `get_validation_errors()` - Detailed error messages for invalid filenames - `is_valid_schema_filename()` - Simple boolean validation - `format_validation_message()` - User-friendly error formatting **Features:** - Regex-based pattern matching - Automatic normalization (spaces → hyphens, lowercase) - Detailed error reporting - Domain validation (must start with letter) - Version validation (major.minor format) ### Comprehensive Test Suite **File:** `tests/test_schema_naming.py` (500+ lines, 50 tests) **Test Coverage:** - ✅ Valid filename variations (simple, hyphenated, with numbers) - ✅ Invalid filenames (wrong extension, missing components, wrong case) - ✅ Filename suggestion with normalization - ✅ Metadata extraction - ✅ Error message generation - ✅ Edge cases (long names, many hyphens, large versions) - ✅ Pattern regex validation **Results:** 50/50 tests passing (100%) ### Specification Document **File:** `roadmap/schema-of-schemas/SCHEMA_NAMING_SPEC.md` **Contents:** - Formal specification of naming convention - Regular expression pattern with explanation - Valid and invalid examples - Version numbering guidelines - Domain naming best practices - Normalization rules - Migration strategy from legacy naming - Implementation guide ## Naming Convention ### Format ``` {domain}-schema-v{major}.{minor}.md ``` ### Examples ``` ✓ manpage-schema-v1.0.md ✓ api-documentation-schema-v1.0.md ✓ terminology-schema-v1.0.md ✓ arc42-schema-v2.1.md ✗ manpage.json (wrong extension) ✗ ManPage-schema-v1.0.md (uppercase) ✗ manpage-v1.0.md (missing 'schema') ✗ manpage-schema-v1.md (missing minor version) ``` ### Components - **domain**: Lowercase, hyphen-separated, starts with letter - **schema**: Literal keyword - **version**: v{major}.{minor} (SemVer simplified) - **extension**: .md (markdown) ## Implementation Highlights ### Automatic Normalization ```python suggest_schema_filename("API Documentation", "2.1") # → "api-documentation-schema-v2.1.md" suggest_schema_filename("My_Custom Type", "1.0") # → "my-custom-type-schema-v1.0.md" ``` ### Detailed Error Reporting ```python format_validation_message("invalid.json") # → Detailed error list + suggested fix ``` ### Metadata Extraction ```python extract_schema_metadata("manpage-schema-v1.0.md") # → {'domain': 'manpage', 'version': '1.0', 'major': 1, 'minor': 0} ``` ## Migration Plan Current schemas will be renamed: ``` Old → New ──────────────────────────────────────────────────────── terminology-schema.json → terminology-schema-v1.0.md api-documentation → api-documentation-schema-v1.0.md enhanced-manpage → manpage-schema-v2.0.md markdown-manpage → DELETE (duplicate) markdown-manpage-schema.json → DELETE (duplicate) ``` ## Phase 1 Status: ✅ COMPLETE ### Completed - [x] Schema naming module implementation - [x] Comprehensive test suite (50 tests, 100% passing) - [x] Specification document - [x] TODO.md updated ### Next: Phase 2 - [ ] Update CLI schema-ingest with validation - [ ] Implement markdown schema loader - [ ] Parse frontmatter and JSON code blocks - [ ] Update SchemaValidator for .md support ## Testing ```bash # Run tests pytest tests/test_schema_naming.py -v # → 50 passed in 0.48s # Test interactively python -c " from markitect.schema_naming import validate_schema_filename print(validate_schema_filename('manpage-schema-v1.0.md')) " # → (True, {'domain': 'manpage', 'version': '1.0', ...}) ``` ## Files Changed - markitect/schema_naming.py (NEW, 380 lines) - tests/test_schema_naming.py (NEW, 500+ lines) - roadmap/schema-of-schemas/SCHEMA_NAMING_SPEC.md (NEW) - TODO.md (updated progress tracking) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-04 23:51:29 +01:00
# Schema Naming Convention Specification
**Version:** 1.0
**Status:** Implemented
**Created:** 2026-01-04
## Overview
This specification defines the filename convention for all MarkiTect schema files to ensure consistency, discoverability, and version tracking across the schema ecosystem.
## Filename Format
### Standard Format
```
{domain}-schema-v{major}.{minor}.md
```
### Components
| Component | Description | Rules | Examples |
|-----------|-------------|-------|----------|
| **domain** | Schema domain identifier | - Lowercase only<br>- Start with letter<br>- Letters, numbers, hyphens<br>- No consecutive hyphens<br>- No leading/trailing hyphens | `manpage`<br>`api-documentation`<br>`arc42` |
| **schema** | Literal keyword | - Must be exactly `schema` | `schema` |
| **version** | SemVer major.minor | - Format: `v{major}.{minor}`<br>- Non-negative integers<br>- Must include both major and minor | `v1.0`<br>`v2.5`<br>`v10.25` |
| **extension** | File extension | - Must be `.md` (markdown) | `.md` |
### Regular Expression
```regex
^[a-z][a-z0-9-]*-schema-v\d+\.\d+\.md$
```
**Breakdown:**
- `^[a-z]` - Start with lowercase letter
- `[a-z0-9-]*` - Followed by lowercase letters, numbers, or hyphens
- `-schema-` - Literal string
- `v\d+\.\d+` - Version (v + digits + dot + digits)
- `\.md$` - Extension
## Valid Examples
### Simple Domains
```
manpage-schema-v1.0.md
terminology-schema-v1.0.md
glossary-schema-v1.0.md
```
### Multi-Word Domains
```
api-documentation-schema-v1.0.md
architecture-decision-record-schema-v1.0.md
software-requirements-specification-schema-v1.0.md
```
### With Numbers
```
arc42-schema-v1.0.md
rfc2119-keywords-schema-v1.0.md
iso27001-schema-v1.0.md
```
### Version Variations
```
manpage-schema-v1.0.md # Initial version
manpage-schema-v1.1.md # Minor update
manpage-schema-v2.0.md # Breaking change
manpage-schema-v10.25.md # Double-digit versions
```
## Invalid Examples
### Wrong Extension
```
❌ manpage-schema-v1.0.json # Must be .md
❌ manpage-schema-v1.0.yaml # Must be .md
❌ manpage-schema-v1.0 # Missing extension
```
### Missing Components
```
❌ manpage-v1.0.md # Missing "schema" keyword
❌ manpage-schema.md # Missing version
❌ manpage.md # Missing "schema" and version
```
### Version Format Errors
```
❌ manpage-schema-1.0.md # Missing 'v' prefix
❌ manpage-schema-v1.md # Missing minor version
❌ manpage-schema-v1.0.0.md # Too many version parts (patch not used)
❌ manpage-schema-v1-0.md # Hyphen instead of dot
```
### Case Errors
```
❌ ManPage-schema-v1.0.md # Uppercase in domain
❌ manpage-Schema-v1.0.md # Uppercase in keyword
❌ MANPAGE-SCHEMA-V1.0.MD # All uppercase
```
### Domain Format Errors
```
❌ 42answers-schema-v1.0.md # Starts with number
❌ -manpage-schema-v1.0.md # Starts with hyphen
❌ man_page-schema-v1.0.md # Underscore (use hyphen)
❌ man page-schema-v1.0.md # Space (use hyphen)
❌ my--schema-v1.0.md # Consecutive hyphens
```
## Version Numbering Guidelines
### Semantic Versioning
We use simplified SemVer with major.minor only:
**Major Version (X.0):**
- Breaking changes to schema structure
- Incompatible with previous version
- Documents validated against v1.0 may fail v2.0
**Examples:**
- `manpage-schema-v1.0.md``manpage-schema-v2.0.md` (breaking change)
- `api-schema-v1.0.md``api-schema-v2.0.md` (new required sections)
**Minor Version (X.Y):**
- Backward-compatible additions
- New optional sections or fields
- Relaxed constraints
- Documents validated against v1.0 still validate against v1.1
**Examples:**
- `manpage-schema-v1.0.md``manpage-schema-v1.1.md` (new optional section)
- `api-schema-v2.0.md``api-schema-v2.1.md` (additional metadata)
### Version Incrementing
```
v1.0 → v1.1 → v1.2 → ... → v1.9 → v1.10 → v1.11
v2.0 (breaking change)
```
### Initial Version
All new schemas start at `v1.0.md`:
```bash
# New schema
my-new-type-schema-v1.0.md
```
## Domain Naming Guidelines
### Good Domain Names
**Descriptive and Specific:**
```
✓ manpage-schema-v1.0.md # Clear: Unix manual pages
✓ api-documentation-schema-v1.0.md # Clear: API docs
✓ architecture-decision-record-schema-v1.0.md # Full ADR name
```
**Concise but Meaningful:**
```
✓ adr-schema-v1.0.md # Common abbreviation
✓ rfc-schema-v1.0.md # Well-known acronym
✓ arc42-schema-v1.0.md # Standard name
```
### Poor Domain Names
**Too Generic:**
```
❌ document-schema-v1.0.md # Too vague
❌ markdown-schema-v1.0.md # All schemas are markdown
❌ schema-schema-v1.0.md # Redundant (use "metaschema")
```
**Too Verbose:**
```
❌ my-custom-documentation-template-for-apis-v1.0.md # Too long
→ api-documentation-schema-v1.0.md # Better
```
**Unclear Abbreviations:**
```
❌ mt-schema-v1.0.md # What is "mt"?
❌ doc-schema-v1.0.md # Too generic
```
## Normalization Rules
When converting arbitrary strings to valid domain names:
1. **Convert to lowercase**
- `API Documentation``api documentation`
2. **Replace separators with hyphens**
- Spaces: `api documentation``api-documentation`
- Underscores: `my_type``my-type`
- Multiple separators: `my type``my--type`
3. **Remove consecutive hyphens**
- `my--type``my-type`
4. **Remove leading/trailing hyphens**
- `-my-type-``my-type`
5. **Validate result**
- Must start with letter
- Only lowercase letters, numbers, hyphens
### Example Normalizations
```python
"API Documentation" → "api-documentation-schema-v1.0.md"
"My_Custom_Type" → "my-custom-type-schema-v1.0.md"
"arc42 Architecture" → "arc42-architecture-schema-v1.0.md"
"--leading-hyphen" → "leading-hyphen-schema-v1.0.md"
```
## Implementation
### Validation Function
The naming convention is enforced by `markitect.schema_naming.validate_schema_filename()`:
```python
from markitect.schema_naming import validate_schema_filename
is_valid, metadata = validate_schema_filename("manpage-schema-v1.0.md")
if is_valid:
print(f"Domain: {metadata['domain']}")
print(f"Version: {metadata['version']}")
print(f"Major: {metadata['major']}, Minor: {metadata['minor']}")
```
### Suggestion Function
Generate valid filenames from arbitrary input:
```python
from markitect.schema_naming import suggest_schema_filename
# From clean input
filename = suggest_schema_filename("manpage", "1.0")
# → "manpage-schema-v1.0.md"
# From messy input (with normalization)
filename = suggest_schema_filename("API Documentation", "2.1")
# → "api-documentation-schema-v1.0.md"
```
### CLI Integration
The `schema-ingest` command validates filenames:
```bash
# Valid filename - accepted
$ markitect schema-ingest manpage-schema-v1.0.md
✅ Schema stored successfully
# Invalid filename - rejected (unless --force)
$ markitect schema-ingest manpage.json
❌ Invalid schema filename: manpage.json
Expected format: {domain}-schema-v{major}.{minor}.md
Example: manpage-schema-v1.0.md
Suggested filename: manpage-schema-v1.0.md
Use --force to skip validation
```
## Migration from Legacy Naming
### Current State Analysis
Existing schemas with inconsistent naming:
```
terminology-schema.json # Has .json extension
api-documentation # No version, no extension
enhanced-manpage # No version, no extension, unclear name
markdown-manpage # No version, no extension
markdown-manpage-schema.json # Has .json extension
```
### Migration Strategy
1. **Identify domain and version**
2. **Apply naming convention**
3. **Update database registration**
4. **Remove legacy entries**
### Migration Mapping
```
Old Name → New Name
────────────────────────────────────────────────────────────────
terminology-schema.json → terminology-schema-v1.0.md
api-documentation → api-documentation-schema-v1.0.md
enhanced-manpage → manpage-schema-v2.0.md
markdown-manpage → (DELETE - duplicate)
markdown-manpage-schema.json → (DELETE - duplicate)
```
**Rationale:**
- `enhanced-manpage` → v2.0 (has breaking changes: classification system)
- `markdown-manpage` variants → DELETE (superseded by v1.0 and v2.0)
## Special Cases
### Metaschema
The schema-for-schemas follows the same convention:
```
schema-schema-v1.0.md
```
Domain is `schema`, indicating it validates schemas themselves.
### Multiple Schemas for Same Domain
Use version numbers to distinguish:
```
manpage-schema-v1.0.md # Original
manpage-schema-v2.0.md # Enhanced with classifications
```
Or use more specific domain names:
```
manpage-simple-schema-v1.0.md # Simplified variant
manpage-extended-schema-v1.0.md # Extended variant
```
## Validation Testing
All schemas should pass the naming convention validation:
```bash
# Test a filename
python -c "
from markitect.schema_naming import is_valid_schema_filename
print(is_valid_schema_filename('manpage-schema-v1.0.md'))
"
# → True
# Get detailed errors
python -c "
from markitect.schema_naming import get_validation_errors
errors = get_validation_errors('invalid.json')
for error in errors:
print(error)
"
```
## Benefits
### Consistency
- All schemas follow same pattern
- Easy to recognize schema files
- Predictable naming
### Versioning
- Clear version tracking
- Multiple versions can coexist
- Breaking changes explicit (major version bump)
### Discoverability
- Glob patterns work: `*-schema-v*.md`
- Easy to list all schemas: `ls *-schema-*.md`
- Domain easily extractable
### Tooling
- Programmatic validation
- Automatic suggestion
- Migration support
## References
- **Implementation:** `markitect/schema_naming.py`
- **Tests:** `tests/test_schema_naming.py`
- **Workplan:** `roadmap/schema-of-schemas/WORKPLAN.md`
- **Examples:** `examples/schemas/manpage-schema-v1.0.md`
## Changelog
### v1.0 (2026-01-04)
- Initial specification
- Implemented validation and suggestion functions
- 50 unit tests (100% passing)
- CLI integration planned