markitect-main/roadmap/schema-of-schemas/SCHEMA_MANAGEMENT_SUMMARY.md

155 lines
4.2 KiB
Markdown
Raw Normal View History

chore: establish schema-of-schemas workplan and reorganize roadmap This commit sets up the comprehensive workplan for implementing a markdown-first schema management system with naming conventions, versioning, and self-validation capabilities. ## Directory Reorganization - Renamed `todo/` → `roadmap/` for better organization - Created `roadmap/schema-of-schemas/` subdirectory - Moved schema management planning artifacts to dedicated directory ## Planning Artifacts Created ### Workplan & Documentation - **WORKPLAN.md** (19KB) - Comprehensive 6-phase implementation plan - **SCHEMA_MANAGEMENT_PROPOSAL.md** - Full analysis with 4 options - **SCHEMA_MANAGEMENT_SUMMARY.md** - Executive summary - **README.md** - Quick reference guide ### Example Schema - **examples/schemas/manpage-schema-v1.md** - Demonstrates markdown format ## Schema Management System Design ### Naming Convention **Format:** `{domain}-schema-v{major}.{minor}.md` **Examples:** - `manpage-schema-v1.0.md` - `terminology-schema-v1.0.md` - `api-documentation-schema-v1.0.md` ### Markdown-First Format Schemas will be markdown files with: - YAML frontmatter for metadata - Rich documentation sections - Embedded JSON schema in code block - Version history and examples ### Implementation Phases (8-10 days) **Phase 0:** Planning & Setup ✅ (0.5 days) - COMPLETE **Phase 1:** Filename Convention (1 day) - NEXT **Phase 2:** Markdown Loader (2-3 days) **Phase 3:** Schema-for-Schemas (2 days) **Phase 4:** Schema Migration (1-2 days) **Phase 5:** CLI & Documentation (1 day) **Phase 6:** Testing & Validation (1 day) ### Goals 1. ✅ Establish naming convention 2. ⏳ Implement filename validation 3. ⏳ Create markdown schema loader 4. ⏳ Build schema-for-schemas metaschema 5. ⏳ Migrate 5 existing schemas (remove 2 duplicates) 6. ⏳ Update CLI and documentation ## Updated Tracking ### TODO.md - Added Schema-of-Schemas as active work item - Documented Phase 1 tasks and timeline - Paused capability extraction work ### CHANGELOG.md - Added schema management system to [Unreleased] - Documented directory reorganization - Added "In Progress" section for current work ## Next Steps Begin Phase 1: 1. Implement schema_naming.py with validation 2. Add unit tests 3. Update CLI schema-ingest command 4. Create naming specification document ## Files Changed - CHANGELOG.md - Added unreleased schema management features - TODO.md - Updated active work tracking - roadmap/ - Reorganized from todo/ - roadmap/schema-of-schemas/ - New planning directory - examples/schemas/ - Example markdown schema 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-04 23:47:02 +01:00
# Schema Management: Executive Summary
**TL;DR:** Implement naming conventions, versioning, and markdown-first schema format to solve current schema management issues.
## Problems Identified
1. **Inconsistent Naming** - Mix of `schema.json` suffix and no suffix
2. **No Versioning** - Can't track schema evolution or maintain multiple versions
3. **Duplicate Schemas** - 3 manpage schemas with similar content
4. **Format Mismatch** - JSON schemas in markdown-centric tool
## Recommended Solution
### 1. Naming Convention (Immediate)
**Format:** `{domain}-schema-v{major}.{minor}.json` or `.md`
**Examples:**
```
manpage-schema-v1.0.json
terminology-schema-v1.0.json
api-documentation-schema-v1.0.json
```
**Migration:**
```
markdown-manpage → manpage-schema-v1.0.json
enhanced-manpage → manpage-schema-v2.0.json (breaking changes)
terminology-schema.json → terminology-schema-v1.0.json
```
### 2. Markdown-First Format (Short-term)
**Proposal:** Store schemas as markdown files with embedded JSON
**Benefits:**
- Aligns with markdown philosophy ✅
- Rich documentation alongside schema ✅
- Version history in same file ✅
- Examples and usage inline ✅
- Lower barrier to entry ✅
**Example:** See `examples/schemas/manpage-schema-v1.md`
**Format:**
```markdown
# Schema Title v1.0
## Documentation sections...
## Schema Definition
\`\`\`json
{ schema here }
\`\`\`
```
### 3. Schema Metadata Standard (Immediate)
**Required fields:**
```json
{
"$schema": "http://json-schema.org/draft-07/schema#",
"$id": "https://markitect.dev/schemas/{domain}/v{major}",
"version": "1.0.0",
"title": "Human Readable Title",
"description": "Detailed description",
"x-markitect-metadata": {
"domain": "manpage",
"document-types": ["manual-page"],
"created": "2026-01-04",
"example": "examples/manpages/example.md"
}
}
```
## Implementation Phases
### Phase 1: Foundation (1-2 days)
- [x] Analyze current state
- [ ] Define naming convention spec
- [ ] Create schema metadata template
- [ ] Rename existing schemas
- [ ] Update schema-catalog.yaml
### Phase 2: Markdown Format (3-5 days)
- [ ] Design markdown schema format
- [ ] Implement parser (extract JSON from markdown)
- [ ] Convert 1 schema as proof-of-concept
- [ ] Test and iterate
- [ ] Migrate all schemas
### Phase 3: Tooling (2-3 days)
- [ ] Update CLI to support .md schemas
- [ ] Add schema validation command
- [ ] Create migration guide
- [ ] Update documentation
## Cost-Benefit Analysis
**Cost:** 6-10 days total effort
**Benefits:**
- Professional schema management ⭐⭐⭐⭐⭐
- Better discoverability ⭐⭐⭐⭐
- Easier maintenance ⭐⭐⭐⭐⭐
- Markdown alignment ⭐⭐⭐⭐⭐
- Version tracking ⭐⭐⭐⭐⭐
**ROI:** High - Foundational improvement that benefits all future schema work
## Alternative Considered
**Alternative:** Keep JSON, generate markdown docs automatically
**Pros:**
- Simpler implementation (2-3 days)
- JSON remains source of truth
- Standard tooling works
**Cons:**
- Doesn't solve format mismatch
- Documentation generated, not authored
- Two files to manage
**Verdict:** Markdown-first better aligns with project philosophy
## Quick Wins (Today)
1. **Rename schemas** with versioned names (30 minutes)
2. **Add metadata** to existing schemas (1 hour)
3. **Update catalog** with proper versioning (30 minutes)
## Questions to Resolve
1. **File extension:** `.md` or `.schema.md` for markdown schemas?
2. **JSON extraction:** Real-time or pre-compiled cache?
3. **Backward compatibility:** Support both formats during transition?
4. **CLI changes:** `--schema file.md` or auto-detect format?
## Next Steps
1. **Review** this proposal and example (examples/schemas/manpage-schema-v1.md)
2. **Decide** on markdown-first vs generated docs approach
3. **Prototype** parser for markdown schemas
4. **Migrate** one schema as proof-of-concept
5. **Iterate** based on feedback
6. **Full rollout** to all schemas
## References
- Full proposal: [SCHEMA_MANAGEMENT_PROPOSAL.md](./SCHEMA_MANAGEMENT_PROPOSAL.md)
- Example markdown schema: [examples/schemas/manpage-schema-v1.md](../../examples/schemas/manpage-schema-v1.md)
- Current schema catalog: [markitect/schemas/schema-catalog.yaml](../../markitect/schemas/schema-catalog.yaml)