diff --git a/SCOPE.md b/SCOPE.md index 1e87258..6ff2543 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -1,32 +1,94 @@ # SCOPE -## Purpose +Reviewed: 2026-09-27 · Scope: ongoing research and a versioned knowledge base. + +## Purpose and deliverable Develop a framework-independent frontend architecture pattern language for human -and agent reference, following the preliminary direction in `INTENT.md`. +and agent reference, following [INTENT.md](INTENT.md). Its intended practical +outcome is an explainable path from a frontend problem and its forces to compatible +patterns and implementation options. + +The deliverable is a versioned Markdown knowledge base with structured metadata, +traceable evidence and transparent revision history. The language evolves through +bounded investigations and review gates; it has no final completion date. Completing +an experiment or milestone does not establish a mature language. ## In scope -- `canon/`: responsibilities, forces, scales, relations and the pattern schema. -- `patterns/`: patterns with context, evidence, trade-offs and failure modes. -- `research/`: sources, interpretations, candidate concepts and counterexamples. -- `sequences/` and `architectures/`: decisions and configurations through patterns. -- `technologies/`: descriptive mappings to implementation options. -- `templates/`: authoring structures and future metadata validation. +- `canon/`: responsibilities, forces, scales, relations and the metadata contract, + including testing whether the proposed vocabulary explains contrasting cases. +- `research/`: identifiable sources, claim-level interpretations, candidate patterns, + counterexamples, pinned implementation studies and bounded executable experiments. +- `patterns/`: evidence-qualified canonical records when admission is justified; + research candidates remain separately identified until then. +- `sequences/` and `architectures/`: decision ordering, compatible configurations, + alternatives and resulting contexts, exercised through contrasting walkthroughs. +- `technologies/`: descriptive mappings from patterns to implementation approaches, + including their constraints and limits. +- `templates/`, `scripts/` and the infospace-bench adapter: authoring, metadata checks + and reproducible research evaluation. Live model execution uses llm-connect. +- `research/evolution/` and `history/`: evidence profiles, immutable snapshots, + prospective questions and effort, supersession, learning outcomes and dated + assessments. Track applicability, trustworthiness, usefulness, coherence, + maintainability and learning efficiency separately. -## Out of scope +These areas include intended capabilities; directory presence is not evidence that +the corresponding capability is implemented or validated. -- Building or operating a frontend application or framework. -- Ranking frameworks or prescribing a universal architecture. -- Treating candidate lists or unsourced interpretations as established evidence. +## Current capabilities and limits -## Current state +As of 2026-09-27, seventeen developed research candidates remain draft hypotheses; +none is canonically admitted. Sources and claim-level reviews, metadata contract +checks, an infospace-bench adapter and bounded runtime research fixtures exist. +Four walkthroughs are synthetic drafts or provisional studies, not validated +architecture recipes. Six further validation cases remain planned. -Research scaffold with an infospace-bench adapter as of 2026-09-26. The adapter -has deterministic integration tests, completed bounded live investigations through -llm-connect, and four synthetic architecture walkthroughs (VC-03 and VC-06 provisional). Seventeen developed research -drafts remain hypotheses; none is canonically admitted. No frontend application runtime exists. See `research/bench/README.md`, -`REVIEW.md` (the initial review) and `workplans/` for follow-up work. +The frozen development pilot scored 18/24 in each presentation condition after +submitted model-review adjudication. It establishes neither a naming benefit nor +equivalence; independent human validation and decision-time measurement remain open. + +Evolution profiles, a baseline, learning ledger, snapshots and a generated report +are implemented. Four effort-accounted prospective cycles belong to four different +comparison groups: there is no comparable window for a diminishing-returns claim. +Evidence types, implementation families and review provenance remain distinct. +Static source mechanisms and authored runtime fixtures do not establish complete +pattern recurrence, production fitness or independent human confirmation. + +Canonical admission, M1/M2 acceptance, demonstrated generativity, full graph export, +held-out retrieval evaluation and developed technology mappings remain open. +There is research runtime code, but no maintained frontend application or framework +is offered as the product. See the [evolution report](research/evolution/EvolutionReport.md) +for current counts and evidence limits; [REVIEW.md](REVIEW.md) preserves the initial +2026-09-23 review and is not a current capability inventory. + +## Research boundaries and exclusions + +- Do not build or operate a general frontend application, framework, router or + release platform as the deliverable. Fixtures exist to investigate claims. +- Do not rank frameworks, prescribe a universal architecture or turn the language + into a component cookbook or collection of tool-specific idioms. +- Do not treat candidate counts, source counts, passing assertions or model agreement + as maturity, independent recurrence or proof of usefulness. +- Do not infer accessibility conformance, production guarantees or upstream defects + from bounded DOM experiments or partial source inspection. +- Do not combine viability dimensions into a maturity percentage, infer historical + effort from commits, or pool incomparable studies to claim learning saturation. + +## Work ownership and review boundaries + +[WP-0002](workplans/FEP-WP-0002-seed-language-validation.md) owns seed evidence, +connected-corpus validation and M1. [WP-0003](workplans/FEP-WP-0003-contrasting-schools.md) +owns contrasting streams, broader walkthroughs, ontology decisions and M2. +[WP-0004](workplans/FEP-WP-0004-reference-surface.md) is proposed follow-up for graph, +held-out retrieval and real-repository applicability, subject to its recorded gates. +[WP-0006](workplans/FEP-WP-0006-language-evolution.md) owns ongoing learning visibility, +review dates and comparable-cycle measurement. Actionable work stays in workplans; +this file describes boundaries, not a second backlog. + +The [2026-09-27 scope–intent assessment](history/2026-09-27T010237+0200-scope-vs-intent.md) +records the current gaps and recommended priorities. Future assessments should +preserve that historical judgment and cite changed evidence. ## Registration diff --git a/TREE.md b/TREE.md index 0bf763b..3a042d5 100644 --- a/TREE.md +++ b/TREE.md @@ -1,6 +1,6 @@ # Repository file inventory -Project files, including pilot receipts and reviews; updated 2026-09-26. +Project files, including pilot receipts and reviews; updated 2026-09-27. ```text .custodian-brief.md @@ -28,6 +28,7 @@ canon/Relations.md canon/Responsibilities.md canon/Scales.md canon/vocabulary.yaml +history/2026-09-27T010237+0200-scope-vs-intent.md patterns/README.md patterns/address/README.md patterns/affordance/README.md diff --git a/history/2026-09-27T010237+0200-scope-vs-intent.md b/history/2026-09-27T010237+0200-scope-vs-intent.md new file mode 100644 index 0000000..a9c0bd6 --- /dev/null +++ b/history/2026-09-27T010237+0200-scope-vs-intent.md @@ -0,0 +1,168 @@ +# SCOPE.md versus INTENT.md — assessment + +Assessment started: 2026-09-27T01:02:37+02:00 (Europe/Berlin), +2026-09-26T23:02:37Z. Assessor: Codex, author-model review; not independent validation. +Evidence baseline: repository commit `aee2971732b2f91b959db70d03f4a146ae63228b`. +Comparison: unchanged [INTENT.md](../INTENT.md) against [SCOPE.md](../SCOPE.md) as +updated in the commit containing this assessment. This is a point-in-time judgment, +not a new empirical research cycle or a canonical-admission decision. + +## Overall assessment + +The updated scope is aligned with the intent: a framework-independent, contextual, +evidence-aware language that evolves indefinitely. There is no need to expand the +product boundary to close the main gaps. The central gap is between the intended +ability to construct an explainable architecture and the demonstrated capability: +we have a substantially stronger research process, but usefulness and generativity +remain unproven. + +The previous scope understated current work: it called metadata validation future +work, omitted evolution measurement and said no frontend runtime existed. Research +fixtures now execute frontend behavior; operating a frontend product remains outside +scope. The update distinguishes these facts, preserves non-goals and makes planned +reference capabilities explicit. It does not promote candidates or close gates. + +## Alignment with the intended properties + +| Intent | Current evidence and scope coverage | Assessment | +| --- | --- | --- | +| Generative and contextual | Seventeen candidates describe forces and consequences; four synthetic/provisional walkthroughs trace decisions. | Contextual authoring exists; reliable construction of architectures is unproven. | +| Relational and multi-scale | Metadata contract and vocabulary exist; candidate connections and walkthrough sequences are provisional. | Connected canonical corpus, validated relation semantics and cross-scale applicability remain open. | +| Evidence-aware | Source identities, claim reviews, pinned static studies, runtime receipts and qualified profiles exist. | Strong process alignment; evidence strength varies and complete recurrence remains unassessed. | +| Technology-independent | Candidates express contracts across several tools; mappings are intended deliverables. | Independence is a design discipline, not yet established across the full range of schools. | +| Agent- and human-readable | Markdown, metadata examples and a research adapter exist. | Retrieval, practitioner usability and benefit from naming/relations remain unvalidated. | +| Evolvable | Baseline, ledger, snapshots, profiles, prospective protocols and deterministic report exist. | Change is transparent; learning trends and maintenance effectiveness are not yet measurable from this sample. | + +## Most relevant gaps, in priority order + +### 1. Demonstrated usefulness and generativity — highest priority + +**Intent:** A person or agent should derive a small, compatible pattern configuration +and map it to implementation options from a problem description. + +**Evidence:** Four walkthroughs remain synthetic/provisional. The frozen pilot's +adjudicated scores are 18/24 named and 18/24 prose. Its submitted review was model +review, not independent human review. Equal totals establish neither equivalence nor +a benefit from the pattern language; decision time is unknown. The six briefs are +consumed development cases, not fresh held-out tasks. + +**Consequence:** More mechanism evidence can improve advice without demonstrating +that the language itself helps architecture decisions. This is the largest distance +from the mission's success condition. + +**Next evidence:** Complete independent review of a small connected configuration, +including rejected alternatives, missed constraints and implementation mapping. +Design fresh, preregistered usefulness tasks at the appropriate gate, measuring +selection quality, unsupported claims and decision effort. Preserve the frozen pilot. +Owners: WP-0002 T03; WP-0003 T02; later WP-0004 T02. + +### 2. Independent recurrence and admission — high priority + +**Intent:** Pattern recommendations should have traceable known uses and justified, +reversible maturity judgments. + +**Evidence:** All 17 candidates remain hypotheses; independent full-contract-use +counts are unassessed for all 17, and independent human profile reviews are 0/17. +The latest twelve-file Zulip study supplies partial static mechanisms and a useful +correction toward effect-specific applicability. It does not execute Zulip, establish +complete pattern recurrence or validate a suspected race. Repeated authored trials +remain one implementation family. + +**Consequence:** A tested mechanism can be mistaken for a reusable pattern if the +pattern's complete context, forces and consequences are not checked elsewhere. + +**Next evidence:** Retain the bounded caller/transport/test follow-up already in +WP-0003 T01, then assess a small seed subset against independent implementations and +explicit admission criteria. Record partial matches, contradictions and unknowns; +retain hypothesis status where needed. M1/M2 reviews must explain judgments rather +than infer them from counts. Owners: WP-0002 T02/T03; WP-0003 T01/T03. + +### 3. Breadth of contexts and conceptual coherence — high priority + +**Intent:** Reconcile architectural schools into a coherent, multi-scale vocabulary. + +**Evidence:** The report records four of ten cases as drafted/provisional and six +planned. Recovery, navigation, focus and release boundaries have received detailed +investigations, while several other candidates have documentary support only. +Coherence metrics are unmeasured; Conflict Copy versus Explicit Reconciliation is +an explicit unresolved overlap. The adequacy of the responsibility model remains an +open intent question. + +**Consequence:** Depth in recent threads may bias the vocabulary toward their forces. +Seventeen names do not establish balanced coverage or that the concepts are distinct. + +**Next evidence:** Choose a contrasting case from the existing validation plan using +the stream coverage gaps, and record which concepts fail, overlap or are missing. +Resolve retain/merge/split/reject questions before expanding terminology solely to +increase coverage counts. Owners: WP-0003 T01/T02/T03, respecting M1 dependencies. + +### 4. Usable relational reference and implementation mappings — medium priority + +**Intent:** Humans and agents should retrieve applicable patterns, conflicts, +prerequisites, resulting contexts and implementation options. + +**Evidence:** Metadata contract checks exist, but no admitted canonical corpus or full +graph export is available; developed technology mappings and held-out retrieval +validation remain future work. WP-0004 is proposed and gated. Early source studies +are valuable but do not complete its later real-repository applicability review. + +**Consequence:** Much of the usable knowledge remains distributed across reviews and +candidate prose, requiring an informed reader to reconstruct the connections. + +**Next evidence:** Use an evidence-qualified connected subset to identify the smallest +useful relation/export and mapping surface. Check unresolved targets and relation +justifications, then evaluate fresh retrieval cases after the recorded gates. Avoid +building a large reference UI before the content contract survives review. +Owners: WP-0002 T03; WP-0004 T01/T02/T03. + +### 5. Comparable learning and maintenance evidence — medium priority + +**Intent:** Make ongoing learning and diminishing returns visible without treating +the language as finished or collapsing viability into a score. + +**Evidence:** Four prospective effort-accounted cycles are in four different comparison +groups. No group has the minimum three comparable cycles for trend review. Fourteen +profiles have unknown last substantive review dates; coherence metrics and decision +time are not measured. Historical effort remains unknown. + +**Consequence:** The process can explain what changed and why, but cannot yet support +claims about saturation, improved efficiency or freshness across the whole language. +A revision-per-hour figure from one short cycle is not an optimization target. + +**Next evidence:** Reconcile review dates from original artifacts; keep unknowns where +necessary. Accumulate comparable cycles only when their questions are useful, retain +corroboration value and assess trends within a stable protocol. Add coherence and +usefulness observations through the reviews above rather than manufacturing counts. +Owner: WP-0006 T03, with usefulness evaluation under WP-0004 T02. + +## Recommended allocation of the next work + +Keep the authored navigation fixture paused. Finish only the already-bounded Zulip +applicability question needed to qualify its source interpretation, then prioritize +independent seed/usefulness review and a contrasting case. Continue lightweight +measurement throughout. This balances broadening with structural improvement: +targeted evidence and coherence work should guide expansion; a large tooling rewrite +or another round of similar fixture tests is not the present bottleneck. + +These recommendations map to existing live or proposed workplans. They do not change +task status, bypass M1/M2, schedule background work or authorize upstream changes. +The language can remain an ongoing learning project while individual reviews close +with explicit evidence and residual questions. + +## Evidence consulted + +- [Evolution report](../research/evolution/EvolutionReport.md): candidate profiles, + case coverage, review provenance, comparison groups and measurement gaps. +- [Pilot adjudication](../research/evaluations/language-pilot-01/submitted-review-01/adjudication.md): + bounded usefulness evidence and reviewer limitations. +- [Navigation thread review](../research/reviews/NavigationThreadReview.md) and + [Zulip navigation study](../research/reviews/ZulipNavigationSourceStudy.md): + fixture pause, independent source contrast and remaining limits. +- [WP-0002](../workplans/FEP-WP-0002-seed-language-validation.md), + [WP-0003](../workplans/FEP-WP-0003-contrasting-schools.md), + [WP-0004](../workplans/FEP-WP-0004-reference-surface.md) and + [WP-0006](../workplans/FEP-WP-0006-language-evolution.md): ownership and gates. +- [Initial review](../REVIEW.md): historical findings, not a current status report. + +Method: local repository review only. No new external source verification, live +inference, runtime experiment or independent human assessment was performed. diff --git a/workplans/FEP-WP-0006-language-evolution.md b/workplans/FEP-WP-0006-language-evolution.md index a8397e0..f2600f1 100644 --- a/workplans/FEP-WP-0006-language-evolution.md +++ b/workplans/FEP-WP-0006-language-evolution.md @@ -130,3 +130,13 @@ defect. T02 retains independent human/AT and offline-case obligations. WP-0006 T retains remaining review dates and comparable-window work; this static protocol is not pooled with runtime cycles. No M1/M2 closure. See research/reviews/ZulipNavigationSourceStudy.md and ZulipNavigationSources.json. + +## Scope–intent assessment — 2026-09-27 + +Updated SCOPE.md to distinguish current research capabilities, intended reference +capabilities and exclusions. The timestamped assessment in +[history/](../history/2026-09-27T010237+0200-scope-vs-intent.md) prioritizes usefulness, +independent recurrence, contrasting coverage/coherence, reference access and +comparable learning evidence. Recommendations map to existing WP-0002/0003/0004/0006 +tasks; statuses and M1/M2 gates are unchanged. This documentation assessment is not +a measured empirical cycle. T03 retains review-date and comparable-window work.