diff --git a/SCOPE.md b/SCOPE.md index 2537807..969d82a 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -26,7 +26,7 @@ Research fixtures exercise bounded mechanisms; they are not implementation recip | Implementation mappings | Locate scoped mechanisms for transactional and offline recovery | [TransactionalRecovery](technologies/TransactionalRecovery.md), [OfflineRecovery](technologies/OfflineRecovery.md); no product supplies a certified complete configuration | | Metadata and publication checks | Validate contract examples, candidate inventory/hashes, classification, evidence locators and immutable release artifacts | `make schema-test evolution-check classification-check reference-check`; checks establish structural traceability, not source truth | | Research bench | Plan and execute bounded source investigations, retain failures, screen structure and explicitly replay compatible cached runs | [Bench adapter](research/bench/README.md); paid inference uses llm-connect and its task authorization/budget, not automatic invocation | -| Learning visibility | Read a selected retrospective ledger, prospective protocols, measured/unknown effort, immutable snapshots and comparison readiness | [ComparisonWindows.json](research/evolution/ComparisonWindows.json); 8 prospective cycles in 7 declared groups, none reaches the 3-cycle review trigger | +| Learning visibility | Read a selected retrospective ledger, prospective protocols, measured/unknown effort, immutable snapshots and comparison readiness | [ComparisonWindows.json](research/evolution/ComparisonWindows.json); 9 prospective attempts/cycles in 8 declared groups (one wholly failed execution), none reaches the 3-cycle review trigger | | Deterministic scheduled reporting | Observe retained natural daily, weekly and October 1 monthly reports from activity-core | [Receipt reconciliation](docs/evidence/2026-10-04-loop-reconciliation.json); zero enrolled consumers, dated owner health snapshot, no admitted pattern-editing executor | The frozen preview and live research are separate version surfaces. Published candidate @@ -44,7 +44,9 @@ paths. Vocabulary and catalog names do not constitute additional implemented pat The consumed development pilot scored 18/24 in each presentation condition after model-review adjudication. It establishes neither a naming benefit nor equivalence. Independent human scores, fresh-scenario replication and decision-time evidence remain -missing. M1/M2 remain revise/defer, and canonical admission is open. Full canonical graph +missing. Fresh atomic replication tooling and a preregistered 12-attempt run exist, +but all attempts failed without answers; [retained record](research/evaluations/language-replication-02/README.md). +M1/M2 remain revise/defer, and canonical admission is open. Full canonical graph export, held-out retrieval evaluation and real-repository applicability studies retain their recorded prerequisites. Eight disclosed preview navigation cases are development checks; they are not held-out retrieval results. @@ -52,7 +54,7 @@ checks; they are not held-out retrieval results. Full-contract recurrence, production fitness, accessibility conformance, native crash and off-device recovery guarantees are not established by the scoped source studies or authored runtime fixtures. Seven prospective cycles have positive recorded research -time; LEARN-0017's time is unknown. A matching group label is not proof of comparability, +time; LEARN-0017 and LEARN-0019 have unknown research time. A matching group label is not proof of comparability, and there is no basis for a diminishing-returns or plateau claim. The capability entry and hosted-publication preparation exist, but hosted federation diff --git a/WORK-RECORDS.md b/WORK-RECORDS.md index babc9c2..3c17a0d 100644 --- a/WORK-RECORDS.md +++ b/WORK-RECORDS.md @@ -16,7 +16,7 @@ | workplan | FEP-WP-0006 | blocked | — | workplans/FEP-WP-0006-language-evolution.md | | workplan | FEP-WP-0007 | finished | — | workplans/FEP-WP-0007-language-v01-preview.md | | workplan | FEP-WP-0008 | blocked | — | workplans/FEP-WP-0008-use-driven-improvement.md | -| workplan | FEP-WP-0009 | backlog | — | workplans/FEP-WP-0009-m1-independent-validation.md | +| workplan | FEP-WP-0009 | blocked | — | workplans/FEP-WP-0009-m1-independent-validation.md | | workplan | FEP-WP-0010 | finished | — | workplans/FEP-WP-0010-reference-consumption.md | | task | FEP-WP-0001-T01 | done | — | workplans/FEP-WP-0001-statehub-bootstrap.md | | task | FEP-WP-0001-T02 | done | — | workplans/FEP-WP-0001-statehub-bootstrap.md | @@ -45,7 +45,7 @@ | task | FEP-WP-0008-T02 | wait | — | workplans/FEP-WP-0008-use-driven-improvement.md | | task | FEP-WP-0008-T03 | wait | — | workplans/FEP-WP-0008-use-driven-improvement.md | | task | FEP-WP-0009-T01 | wait | — | workplans/FEP-WP-0009-m1-independent-validation.md | -| task | FEP-WP-0009-T02 | todo | — | workplans/FEP-WP-0009-m1-independent-validation.md | +| task | FEP-WP-0009-T02 | wait | — | workplans/FEP-WP-0009-m1-independent-validation.md | | task | FEP-WP-0009-T03 | wait | — | workplans/FEP-WP-0009-m1-independent-validation.md | | task | FEP-WP-0010-T01 | done | — | workplans/FEP-WP-0010-reference-consumption.md | | task | FEP-WP-0010-T02 | done | — | workplans/FEP-WP-0010-reference-consumption.md | diff --git a/history/2026-10-04T111438+0200-gap-implementation-and-replication.md b/history/2026-10-04T111438+0200-gap-implementation-and-replication.md new file mode 100644 index 0000000..0a5bf5c --- /dev/null +++ b/history/2026-10-04T111438+0200-gap-implementation-and-replication.md @@ -0,0 +1,32 @@ +# Gap implementation and authorized replication — 2026-10-04 + +All assessed gaps already have live workplan owners: real consumer outcomes WP-0008; +independent validation/fresh replication WP-0009; broader applicability WP-0003/0004; +comparable learning WP-0006. No duplicate workplan was registered. + +Implemented and preregistered the [atomic replication packet and runner](../research/evaluations/language-replication-02/README.md), +with isolation, guidance equivalence, immutable input checks, attempt reservation, +no retries and evidence-bearing scoring. Six fresh authored scenarios have six +binary criteria per condition. The user authorized twelve capped calls; all twelve +attempts failed without answers. Host/sandbox health checks found llm-connect unreachable. +All receipts are retained; scores stay null and six pairs are excluded. T02 waits +for restored service and a newly authorized registered run. No provider billing +claim, naming conclusion, human review or maturity advancement is inferred. + +A [bounded consumer discovery](../docs/evidence/2026-10-04-consumer-discovery.json) +found no indexed eligible external task in the searched active/ready subset. This +is not a global absence proof. WP-0008 retains discovery, enrollment and actual +linked decisions/outcomes. The local consumption bridge remains available. + +Most relevant next gaps remain real consumer decisions/outcomes and independent +human validation. Fresh replication additionally requires working llm-connect; +stronger architecture/application claims and comparable returns remain downstream. +LEARN-0019 retains the failed cycle and unknown research effort. Existing native +workplans retain every residual; frozen pilot/reference evidence is unchanged. + + +Verification: all 120 repository tests passed before the authorized run. Final +ledger/report checks and 22 evolution plus 16 replication tests passed. The commit +guard regression test was corrected to accept either an absent Git path (before +publication) or a byte mismatch (after publication); both prevent adapter construction. +The frozen runner/input bytes remain unchanged. Whitespace checks passed. diff --git a/research/evaluations/language-replication-02/README.md b/research/evaluations/language-replication-02/README.md index a5e33c1..1c8cbe4 100644 --- a/research/evaluations/language-replication-02/README.md +++ b/research/evaluations/language-replication-02/README.md @@ -42,3 +42,27 @@ correctness or independently authenticate a declared reviewer. Keep initial rati disagreements and adjudication. A failed pair is excluded from quality counts, and no score automatically completes human review or M1. Actual execution/editorial outcomes are recorded here after the authorized run; independent-human scores remain pending. + +## Authorized execution outcome — 2026-10-04 + +Preregistration commit: `1e2741a95909341441da05f86a4b6721eae29b60`, synced before +execution. Twelve attempts were reserved and made once each; **all failed**, with +no retained answers. [Raw records](raw/) retain every reservation and receipt. +[Execution summary](execution-summary.json) records elapsed request time and failed +health checks inside/outside the sandbox. The runner records `InfospaceError`, not +its subtype; endpoint unavailability was verified afterward. Provider execution, +token usage and billing are unknown. No retries were made. + +[Availability review](availability-review.json) marks all answers unavailable and +all atomic ratings null. No masked answer review occurred; reading the first receipt +exposed its assignment, which is disclosed. [Comparison summary](comparison-summary.json) +excludes all six pairs. Its zero totals have zero denominators and are inventory +counts, not quality scores or evidence of equivalence. Neither presentation has an +observed result. Original pilot-01 evidence remains unchanged. + +The packet and tested execution/scoring tools are usable, but WP-0009 T02 is waiting +for a usable fresh replication. Do not reuse the reserved run identities or modify +the frozen packet. Any future execution needs service restoration, a new preregistered +run identity and explicit fresh authorization. Human validation remains WP-0009 T01; +consumer discovery/integration remains WP-0008. LEARN-0019 retains this failure rather +than dropping it from research history; it contributes no pattern-quality conclusion. diff --git a/research/evaluations/language-replication-02/availability-review.json b/research/evaluations/language-replication-02/availability-review.json new file mode 100644 index 0000000..a56e1f7 --- /dev/null +++ b/research/evaluations/language-replication-02/availability-review.json @@ -0,0 +1,564 @@ +{ + "schema_version": 1, + "reviewer": { + "name": "Codex", + "role": "author-model", + "independent_human": false, + "assignment_seen": true, + "prior_scores_seen": false + }, + "answers": { + "R02-S01-U": { + "available": false, + "ratings": { + "A1": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A2": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A3": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A4": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A5": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A6": { + "value": null, + "quote": null, + "locator": null, + "reason": null + } + }, + "unsupported_claims": null, + "ambiguity": [ + "No retained answer: generation attempt failed. Criteria and unsupported claims remain unreviewed, not zero." + ], + "suspected_condition_clues": [] + }, + "R02-S01-V": { + "available": false, + "ratings": { + "A1": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A2": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A3": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A4": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A5": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A6": { + "value": null, + "quote": null, + "locator": null, + "reason": null + } + }, + "unsupported_claims": null, + "ambiguity": [ + "No retained answer: generation attempt failed. Criteria and unsupported claims remain unreviewed, not zero." + ], + "suspected_condition_clues": [] + }, + "R02-S02-U": { + "available": false, + "ratings": { + "A1": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A2": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A3": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A4": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A5": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A6": { + "value": null, + "quote": null, + "locator": null, + "reason": null + } + }, + "unsupported_claims": null, + "ambiguity": [ + "No retained answer: generation attempt failed. Criteria and unsupported claims remain unreviewed, not zero." + ], + "suspected_condition_clues": [] + }, + "R02-S02-V": { + "available": false, + "ratings": { + "A1": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A2": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A3": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A4": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A5": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A6": { + "value": null, + "quote": null, + "locator": null, + "reason": null + } + }, + "unsupported_claims": null, + "ambiguity": [ + "No retained answer: generation attempt failed. Criteria and unsupported claims remain unreviewed, not zero." + ], + "suspected_condition_clues": [] + }, + "R02-S03-U": { + "available": false, + "ratings": { + "A1": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A2": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A3": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A4": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A5": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A6": { + "value": null, + "quote": null, + "locator": null, + "reason": null + } + }, + "unsupported_claims": null, + "ambiguity": [ + "No retained answer: generation attempt failed. Criteria and unsupported claims remain unreviewed, not zero." + ], + "suspected_condition_clues": [] + }, + "R02-S03-V": { + "available": false, + "ratings": { + "A1": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A2": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A3": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A4": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A5": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A6": { + "value": null, + "quote": null, + "locator": null, + "reason": null + } + }, + "unsupported_claims": null, + "ambiguity": [ + "No retained answer: generation attempt failed. Criteria and unsupported claims remain unreviewed, not zero." + ], + "suspected_condition_clues": [] + }, + "R02-S04-U": { + "available": false, + "ratings": { + "A1": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A2": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A3": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A4": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A5": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A6": { + "value": null, + "quote": null, + "locator": null, + "reason": null + } + }, + "unsupported_claims": null, + "ambiguity": [ + "No retained answer: generation attempt failed. Criteria and unsupported claims remain unreviewed, not zero." + ], + "suspected_condition_clues": [] + }, + "R02-S04-V": { + "available": false, + "ratings": { + "A1": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A2": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A3": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A4": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A5": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A6": { + "value": null, + "quote": null, + "locator": null, + "reason": null + } + }, + "unsupported_claims": null, + "ambiguity": [ + "No retained answer: generation attempt failed. Criteria and unsupported claims remain unreviewed, not zero." + ], + "suspected_condition_clues": [] + }, + "R02-S05-U": { + "available": false, + "ratings": { + "A1": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A2": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A3": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A4": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A5": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A6": { + "value": null, + "quote": null, + "locator": null, + "reason": null + } + }, + "unsupported_claims": null, + "ambiguity": [ + "No retained answer: generation attempt failed. Criteria and unsupported claims remain unreviewed, not zero." + ], + "suspected_condition_clues": [] + }, + "R02-S05-V": { + "available": false, + "ratings": { + "A1": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A2": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A3": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A4": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A5": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A6": { + "value": null, + "quote": null, + "locator": null, + "reason": null + } + }, + "unsupported_claims": null, + "ambiguity": [ + "No retained answer: generation attempt failed. Criteria and unsupported claims remain unreviewed, not zero." + ], + "suspected_condition_clues": [] + }, + "R02-S06-U": { + "available": false, + "ratings": { + "A1": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A2": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A3": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A4": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A5": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A6": { + "value": null, + "quote": null, + "locator": null, + "reason": null + } + }, + "unsupported_claims": null, + "ambiguity": [ + "No retained answer: generation attempt failed. Criteria and unsupported claims remain unreviewed, not zero." + ], + "suspected_condition_clues": [] + }, + "R02-S06-V": { + "available": false, + "ratings": { + "A1": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A2": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A3": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A4": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A5": { + "value": null, + "quote": null, + "locator": null, + "reason": null + }, + "A6": { + "value": null, + "quote": null, + "locator": null, + "reason": null + } + }, + "unsupported_claims": null, + "ambiguity": [ + "No retained answer: generation attempt failed. Criteria and unsupported claims remain unreviewed, not zero." + ], + "suspected_condition_clues": [] + } + } +} diff --git a/research/evaluations/language-replication-02/comparison-summary.json b/research/evaluations/language-replication-02/comparison-summary.json new file mode 100644 index 0000000..3c3bdae --- /dev/null +++ b/research/evaluations/language-replication-02/comparison-summary.json @@ -0,0 +1,60 @@ +{ + "schema_version": 1, + "reviewer": { + "name": "Codex", + "role": "author-model", + "independent_human": false, + "assignment_seen": true, + "prior_scores_seen": false + }, + "totals": { + "patterns": { + "completed_pairs": 0, + "fulfilled": 0, + "unsafe": 0, + "criteria": 0 + }, + "prose": { + "completed_pairs": 0, + "fulfilled": 0, + "unsafe": 0, + "criteria": 0 + } + }, + "pairs": [ + { + "scenario": "R02-S01", + "included": false, + "results": {} + }, + { + "scenario": "R02-S02", + "included": false, + "results": {} + }, + { + "scenario": "R02-S03", + "included": false, + "results": {} + }, + { + "scenario": "R02-S04", + "included": false, + "results": {} + }, + { + "scenario": "R02-S05", + "included": false, + "results": {} + }, + { + "scenario": "R02-S06", + "included": false, + "results": {} + } + ], + "limits": [ + "Descriptive author-created replication, not proof of superiority/equivalence.", + "Reviewer role is declared provenance; no automatic independent-human or M1 claim." + ] +} diff --git a/research/evaluations/language-replication-02/execution-summary.json b/research/evaluations/language-replication-02/execution-summary.json new file mode 100644 index 0000000..fab113c --- /dev/null +++ b/research/evaluations/language-replication-02/execution-summary.json @@ -0,0 +1,19 @@ +{ + "schema_version": 1, + "date": "2026-10-04", + "attempts_reserved": 12, + "completed": 0, + "failed": 12, + "request_elapsed_seconds": 0.059, + "provider_usage_known": false, + "cost_usd": null, + "health_probe": { + "endpoint": "http://127.0.0.1:18081/health", + "sandbox_exit_code": 7, + "host_exit_code": 7, + "result": "Connection refused/unreachable; no inference retry performed" + }, + "diagnostic_limit": "Frozen runner retains exception class InfospaceError without subtype; health probes establish endpoint unavailability at inspection, not a provider-side error or proven billing outcome.", + "preregistration_commit": "1e2741a95909341441da05f86a4b6721eae29b60", + "no_retries": true +} diff --git a/research/evaluations/language-replication-02/raw/r01/receipt.json b/research/evaluations/language-replication-02/raw/r01/receipt.json new file mode 100644 index 0000000..1a21b30 --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r01/receipt.json @@ -0,0 +1,24 @@ +{ + "run": "r01", + "scenario": "R02-S01", + "condition": "patterns", + "blind_id": "R02-S01-U", + "attempts_reserved": 1, + "prompt_sha256": "91a50e9c1bcb85b650ca7c51155d7c8aed3d8a95f0375a3c87fbcc735b10dde4", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400, + "endpoint": "http://127.0.0.1:18081", + "config": { + "model_name": "custodian-triage-balanced", + "temperature": 0.2, + "max_tokens": 1400, + "model_params": {}, + "max_depth": 3, + "skip_if_exists": true, + "timeout_seconds": 300 + }, + "status": "failed", + "error_type": "InfospaceError", + "elapsed_seconds": 0.053, + "responses": [] +} diff --git a/research/evaluations/language-replication-02/raw/r01/started.json b/research/evaluations/language-replication-02/raw/r01/started.json new file mode 100644 index 0000000..c68f359 --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r01/started.json @@ -0,0 +1,10 @@ +{ + "run": "r01", + "scenario": "R02-S01", + "condition": "patterns", + "blind_id": "R02-S01-U", + "attempts_reserved": 1, + "prompt_sha256": "91a50e9c1bcb85b650ca7c51155d7c8aed3d8a95f0375a3c87fbcc735b10dde4", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400 +} diff --git a/research/evaluations/language-replication-02/raw/r02/receipt.json b/research/evaluations/language-replication-02/raw/r02/receipt.json new file mode 100644 index 0000000..2927081 --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r02/receipt.json @@ -0,0 +1,24 @@ +{ + "run": "r02", + "scenario": "R02-S01", + "condition": "prose", + "blind_id": "R02-S01-V", + "attempts_reserved": 1, + "prompt_sha256": "146a74fc4e3adafd8039553a3676102132551faa8168a5a8fcb401d2b205272c", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400, + "endpoint": "http://127.0.0.1:18081", + "config": { + "model_name": "custodian-triage-balanced", + "temperature": 0.2, + "max_tokens": 1400, + "model_params": {}, + "max_depth": 3, + "skip_if_exists": true, + "timeout_seconds": 300 + }, + "status": "failed", + "error_type": "InfospaceError", + "elapsed_seconds": 0.001, + "responses": [] +} diff --git a/research/evaluations/language-replication-02/raw/r02/started.json b/research/evaluations/language-replication-02/raw/r02/started.json new file mode 100644 index 0000000..5ab6ca8 --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r02/started.json @@ -0,0 +1,10 @@ +{ + "run": "r02", + "scenario": "R02-S01", + "condition": "prose", + "blind_id": "R02-S01-V", + "attempts_reserved": 1, + "prompt_sha256": "146a74fc4e3adafd8039553a3676102132551faa8168a5a8fcb401d2b205272c", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400 +} diff --git a/research/evaluations/language-replication-02/raw/r03/receipt.json b/research/evaluations/language-replication-02/raw/r03/receipt.json new file mode 100644 index 0000000..3b1e174 --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r03/receipt.json @@ -0,0 +1,24 @@ +{ + "run": "r03", + "scenario": "R02-S02", + "condition": "prose", + "blind_id": "R02-S02-V", + "attempts_reserved": 1, + "prompt_sha256": "be924ba36625b663e3957b29d30303b8042f5a382cb70b11ea111a2762635dc3", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400, + "endpoint": "http://127.0.0.1:18081", + "config": { + "model_name": "custodian-triage-balanced", + "temperature": 0.2, + "max_tokens": 1400, + "model_params": {}, + "max_depth": 3, + "skip_if_exists": true, + "timeout_seconds": 300 + }, + "status": "failed", + "error_type": "InfospaceError", + "elapsed_seconds": 0.0, + "responses": [] +} diff --git a/research/evaluations/language-replication-02/raw/r03/started.json b/research/evaluations/language-replication-02/raw/r03/started.json new file mode 100644 index 0000000..d054e78 --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r03/started.json @@ -0,0 +1,10 @@ +{ + "run": "r03", + "scenario": "R02-S02", + "condition": "prose", + "blind_id": "R02-S02-V", + "attempts_reserved": 1, + "prompt_sha256": "be924ba36625b663e3957b29d30303b8042f5a382cb70b11ea111a2762635dc3", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400 +} diff --git a/research/evaluations/language-replication-02/raw/r04/receipt.json b/research/evaluations/language-replication-02/raw/r04/receipt.json new file mode 100644 index 0000000..5f9f872 --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r04/receipt.json @@ -0,0 +1,24 @@ +{ + "run": "r04", + "scenario": "R02-S02", + "condition": "patterns", + "blind_id": "R02-S02-U", + "attempts_reserved": 1, + "prompt_sha256": "3371ff49a2d0f29ac8a65a768f72aebf0228fb5bf950494878ec4980c8b1ff70", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400, + "endpoint": "http://127.0.0.1:18081", + "config": { + "model_name": "custodian-triage-balanced", + "temperature": 0.2, + "max_tokens": 1400, + "model_params": {}, + "max_depth": 3, + "skip_if_exists": true, + "timeout_seconds": 300 + }, + "status": "failed", + "error_type": "InfospaceError", + "elapsed_seconds": 0.001, + "responses": [] +} diff --git a/research/evaluations/language-replication-02/raw/r04/started.json b/research/evaluations/language-replication-02/raw/r04/started.json new file mode 100644 index 0000000..389e154 --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r04/started.json @@ -0,0 +1,10 @@ +{ + "run": "r04", + "scenario": "R02-S02", + "condition": "patterns", + "blind_id": "R02-S02-U", + "attempts_reserved": 1, + "prompt_sha256": "3371ff49a2d0f29ac8a65a768f72aebf0228fb5bf950494878ec4980c8b1ff70", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400 +} diff --git a/research/evaluations/language-replication-02/raw/r05/receipt.json b/research/evaluations/language-replication-02/raw/r05/receipt.json new file mode 100644 index 0000000..3860c62 --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r05/receipt.json @@ -0,0 +1,24 @@ +{ + "run": "r05", + "scenario": "R02-S03", + "condition": "patterns", + "blind_id": "R02-S03-U", + "attempts_reserved": 1, + "prompt_sha256": "7a61c90e25d342750b3d9651c90b157975e441822d66e826b1315bce957cafe3", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400, + "endpoint": "http://127.0.0.1:18081", + "config": { + "model_name": "custodian-triage-balanced", + "temperature": 0.2, + "max_tokens": 1400, + "model_params": {}, + "max_depth": 3, + "skip_if_exists": true, + "timeout_seconds": 300 + }, + "status": "failed", + "error_type": "InfospaceError", + "elapsed_seconds": 0.001, + "responses": [] +} diff --git a/research/evaluations/language-replication-02/raw/r05/started.json b/research/evaluations/language-replication-02/raw/r05/started.json new file mode 100644 index 0000000..e90498e --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r05/started.json @@ -0,0 +1,10 @@ +{ + "run": "r05", + "scenario": "R02-S03", + "condition": "patterns", + "blind_id": "R02-S03-U", + "attempts_reserved": 1, + "prompt_sha256": "7a61c90e25d342750b3d9651c90b157975e441822d66e826b1315bce957cafe3", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400 +} diff --git a/research/evaluations/language-replication-02/raw/r06/receipt.json b/research/evaluations/language-replication-02/raw/r06/receipt.json new file mode 100644 index 0000000..8dde552 --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r06/receipt.json @@ -0,0 +1,24 @@ +{ + "run": "r06", + "scenario": "R02-S03", + "condition": "prose", + "blind_id": "R02-S03-V", + "attempts_reserved": 1, + "prompt_sha256": "76ca0bca70f535da292885af2ead28162daac79e5417b1f92aa68a3d024b7145", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400, + "endpoint": "http://127.0.0.1:18081", + "config": { + "model_name": "custodian-triage-balanced", + "temperature": 0.2, + "max_tokens": 1400, + "model_params": {}, + "max_depth": 3, + "skip_if_exists": true, + "timeout_seconds": 300 + }, + "status": "failed", + "error_type": "InfospaceError", + "elapsed_seconds": 0.001, + "responses": [] +} diff --git a/research/evaluations/language-replication-02/raw/r06/started.json b/research/evaluations/language-replication-02/raw/r06/started.json new file mode 100644 index 0000000..a0e668b --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r06/started.json @@ -0,0 +1,10 @@ +{ + "run": "r06", + "scenario": "R02-S03", + "condition": "prose", + "blind_id": "R02-S03-V", + "attempts_reserved": 1, + "prompt_sha256": "76ca0bca70f535da292885af2ead28162daac79e5417b1f92aa68a3d024b7145", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400 +} diff --git a/research/evaluations/language-replication-02/raw/r07/receipt.json b/research/evaluations/language-replication-02/raw/r07/receipt.json new file mode 100644 index 0000000..7922839 --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r07/receipt.json @@ -0,0 +1,24 @@ +{ + "run": "r07", + "scenario": "R02-S04", + "condition": "prose", + "blind_id": "R02-S04-U", + "attempts_reserved": 1, + "prompt_sha256": "d2e369f6381d6bcc769234a870b079eadec29589308c4ad6a8bc4d17ab035311", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400, + "endpoint": "http://127.0.0.1:18081", + "config": { + "model_name": "custodian-triage-balanced", + "temperature": 0.2, + "max_tokens": 1400, + "model_params": {}, + "max_depth": 3, + "skip_if_exists": true, + "timeout_seconds": 300 + }, + "status": "failed", + "error_type": "InfospaceError", + "elapsed_seconds": 0.0, + "responses": [] +} diff --git a/research/evaluations/language-replication-02/raw/r07/started.json b/research/evaluations/language-replication-02/raw/r07/started.json new file mode 100644 index 0000000..3fb9e3e --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r07/started.json @@ -0,0 +1,10 @@ +{ + "run": "r07", + "scenario": "R02-S04", + "condition": "prose", + "blind_id": "R02-S04-U", + "attempts_reserved": 1, + "prompt_sha256": "d2e369f6381d6bcc769234a870b079eadec29589308c4ad6a8bc4d17ab035311", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400 +} diff --git a/research/evaluations/language-replication-02/raw/r08/receipt.json b/research/evaluations/language-replication-02/raw/r08/receipt.json new file mode 100644 index 0000000..54b740f --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r08/receipt.json @@ -0,0 +1,24 @@ +{ + "run": "r08", + "scenario": "R02-S04", + "condition": "patterns", + "blind_id": "R02-S04-V", + "attempts_reserved": 1, + "prompt_sha256": "8705d35a4bc045405986796e6dcd96d687a82b409e6d0208e697bea5c5d207d1", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400, + "endpoint": "http://127.0.0.1:18081", + "config": { + "model_name": "custodian-triage-balanced", + "temperature": 0.2, + "max_tokens": 1400, + "model_params": {}, + "max_depth": 3, + "skip_if_exists": true, + "timeout_seconds": 300 + }, + "status": "failed", + "error_type": "InfospaceError", + "elapsed_seconds": 0.0, + "responses": [] +} diff --git a/research/evaluations/language-replication-02/raw/r08/started.json b/research/evaluations/language-replication-02/raw/r08/started.json new file mode 100644 index 0000000..6e6c219 --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r08/started.json @@ -0,0 +1,10 @@ +{ + "run": "r08", + "scenario": "R02-S04", + "condition": "patterns", + "blind_id": "R02-S04-V", + "attempts_reserved": 1, + "prompt_sha256": "8705d35a4bc045405986796e6dcd96d687a82b409e6d0208e697bea5c5d207d1", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400 +} diff --git a/research/evaluations/language-replication-02/raw/r09/receipt.json b/research/evaluations/language-replication-02/raw/r09/receipt.json new file mode 100644 index 0000000..c4b0f3b --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r09/receipt.json @@ -0,0 +1,24 @@ +{ + "run": "r09", + "scenario": "R02-S05", + "condition": "patterns", + "blind_id": "R02-S05-U", + "attempts_reserved": 1, + "prompt_sha256": "8944e9bd64680ed927ece8c8e1ad9104b589f5e6b21fa7872ec35e5d728d90c1", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400, + "endpoint": "http://127.0.0.1:18081", + "config": { + "model_name": "custodian-triage-balanced", + "temperature": 0.2, + "max_tokens": 1400, + "model_params": {}, + "max_depth": 3, + "skip_if_exists": true, + "timeout_seconds": 300 + }, + "status": "failed", + "error_type": "InfospaceError", + "elapsed_seconds": 0.001, + "responses": [] +} diff --git a/research/evaluations/language-replication-02/raw/r09/started.json b/research/evaluations/language-replication-02/raw/r09/started.json new file mode 100644 index 0000000..b8769e7 --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r09/started.json @@ -0,0 +1,10 @@ +{ + "run": "r09", + "scenario": "R02-S05", + "condition": "patterns", + "blind_id": "R02-S05-U", + "attempts_reserved": 1, + "prompt_sha256": "8944e9bd64680ed927ece8c8e1ad9104b589f5e6b21fa7872ec35e5d728d90c1", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400 +} diff --git a/research/evaluations/language-replication-02/raw/r10/receipt.json b/research/evaluations/language-replication-02/raw/r10/receipt.json new file mode 100644 index 0000000..c71800a --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r10/receipt.json @@ -0,0 +1,24 @@ +{ + "run": "r10", + "scenario": "R02-S05", + "condition": "prose", + "blind_id": "R02-S05-V", + "attempts_reserved": 1, + "prompt_sha256": "734d42b631b58b06246db211270c10823100e5e3249964aad286dd9e725d725c", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400, + "endpoint": "http://127.0.0.1:18081", + "config": { + "model_name": "custodian-triage-balanced", + "temperature": 0.2, + "max_tokens": 1400, + "model_params": {}, + "max_depth": 3, + "skip_if_exists": true, + "timeout_seconds": 300 + }, + "status": "failed", + "error_type": "InfospaceError", + "elapsed_seconds": 0.0, + "responses": [] +} diff --git a/research/evaluations/language-replication-02/raw/r10/started.json b/research/evaluations/language-replication-02/raw/r10/started.json new file mode 100644 index 0000000..0a9ed3a --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r10/started.json @@ -0,0 +1,10 @@ +{ + "run": "r10", + "scenario": "R02-S05", + "condition": "prose", + "blind_id": "R02-S05-V", + "attempts_reserved": 1, + "prompt_sha256": "734d42b631b58b06246db211270c10823100e5e3249964aad286dd9e725d725c", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400 +} diff --git a/research/evaluations/language-replication-02/raw/r11/receipt.json b/research/evaluations/language-replication-02/raw/r11/receipt.json new file mode 100644 index 0000000..942309c --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r11/receipt.json @@ -0,0 +1,24 @@ +{ + "run": "r11", + "scenario": "R02-S06", + "condition": "prose", + "blind_id": "R02-S06-U", + "attempts_reserved": 1, + "prompt_sha256": "c2007173765706f16ed61ff93f1246b960ca31b3cee5b6e511863606f23a6433", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400, + "endpoint": "http://127.0.0.1:18081", + "config": { + "model_name": "custodian-triage-balanced", + "temperature": 0.2, + "max_tokens": 1400, + "model_params": {}, + "max_depth": 3, + "skip_if_exists": true, + "timeout_seconds": 300 + }, + "status": "failed", + "error_type": "InfospaceError", + "elapsed_seconds": 0.0, + "responses": [] +} diff --git a/research/evaluations/language-replication-02/raw/r11/started.json b/research/evaluations/language-replication-02/raw/r11/started.json new file mode 100644 index 0000000..fd75b9b --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r11/started.json @@ -0,0 +1,10 @@ +{ + "run": "r11", + "scenario": "R02-S06", + "condition": "prose", + "blind_id": "R02-S06-U", + "attempts_reserved": 1, + "prompt_sha256": "c2007173765706f16ed61ff93f1246b960ca31b3cee5b6e511863606f23a6433", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400 +} diff --git a/research/evaluations/language-replication-02/raw/r12/receipt.json b/research/evaluations/language-replication-02/raw/r12/receipt.json new file mode 100644 index 0000000..70749dd --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r12/receipt.json @@ -0,0 +1,24 @@ +{ + "run": "r12", + "scenario": "R02-S06", + "condition": "patterns", + "blind_id": "R02-S06-V", + "attempts_reserved": 1, + "prompt_sha256": "b4c71336ea9dd8563db6432e00f1344ebc3379258364f390af73854df89ab432", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400, + "endpoint": "http://127.0.0.1:18081", + "config": { + "model_name": "custodian-triage-balanced", + "temperature": 0.2, + "max_tokens": 1400, + "model_params": {}, + "max_depth": 3, + "skip_if_exists": true, + "timeout_seconds": 300 + }, + "status": "failed", + "error_type": "InfospaceError", + "elapsed_seconds": 0.001, + "responses": [] +} diff --git a/research/evaluations/language-replication-02/raw/r12/started.json b/research/evaluations/language-replication-02/raw/r12/started.json new file mode 100644 index 0000000..d129305 --- /dev/null +++ b/research/evaluations/language-replication-02/raw/r12/started.json @@ -0,0 +1,10 @@ +{ + "run": "r12", + "scenario": "R02-S06", + "condition": "patterns", + "blind_id": "R02-S06-V", + "attempts_reserved": 1, + "prompt_sha256": "b4c71336ea9dd8563db6432e00f1344ebc3379258364f390af73854df89ab432", + "freeze_sha256": "c674382c1f6399e437794aaaab955168cabe4533ec68b8d4e82af41471d32975", + "max_output_tokens": 1400 +} diff --git a/research/evolution/ComparisonWindows.json b/research/evolution/ComparisonWindows.json index 2290854..d4e21e8 100644 --- a/research/evolution/ComparisonWindows.json +++ b/research/evolution/ComparisonWindows.json @@ -194,6 +194,32 @@ "rejection": 0, "revision": 1 } + }, + { + "stream": "cross-stream", + "comparison_group": "presentation-atomic-replication-v2", + "cycle_ids": [ + "LEARN-0019" + ], + "protocol_refs": [ + "research/evolution/cycles/2026-10-04-presentation-replication.md" + ], + "cycles_until_review_trigger": 2, + "qualitative_review_due": false, + "positive_effort_cycle_ids": [], + "unknown_effort_cycle_ids": [ + "LEARN-0019" + ], + "zero_effort_cycle_ids": [], + "effort_complete": false, + "effort_comparison_ready": false, + "outcomes": { + "corroboration": 0, + "inconclusive": 1, + "merge": 0, + "rejection": 0, + "revision": 0 + } } ] } diff --git a/research/evolution/EvolutionReport.md b/research/evolution/EvolutionReport.md index c8621f4..bd342d4 100644 --- a/research/evolution/EvolutionReport.md +++ b/research/evolution/EvolutionReport.md @@ -52,12 +52,12 @@ Ten selected retrospective outcomes (not a historical census), followed by prosp | Outcome | Entries (not independent discoveries) | | --- | ---: | | corroboration | 5 | -| inconclusive | 0 | +| inconclusive | 1 | | merge | 0 | | rejection | 0 | | revision | 13 | -Cycles with recorded research effort: 7/18. Unknown effort is neither zero nor estimated from commits, tokens or assertion counts. +Cycles with recorded research effort: 7/19. Unknown effort is neither zero nor estimated from commits, tokens or assertion counts. ### Comparison readiness @@ -72,6 +72,7 @@ Declared groups require a review of shared evidence tier, question granularity, | F / F-navigation-ownership-v1 | 1 | 1 | 0 / 0 | pending | not ready | | cross-stream / CD-acceptance-projection-static-v1 | 1 | 1 | 0 / 0 | pending | not ready | | cross-stream / CD-authority-projection-documentary-v1 | 1 | 1 | 0 / 0 | pending | not ready | +| cross-stream / presentation-atomic-replication-v2 | 1 | 0 | 1 / 0 | pending | not ready | Unknown or zero effort is not dropped to create a favorable window. Qualitative review can retain those cycles, but effort comparisons need an explicitly reviewed window with measured positive research time. No trend or plateau is inferred. @@ -132,6 +133,14 @@ One bounded question counts at most one consequential outcome. Corroboration and One bounded question counts at most one consequential outcome. Corroboration and inconclusive work remain visible separately. Fewer than three declared cycles: qualitative review trigger pending; no trend assessment. +### Stream cross-stream / presentation-atomic-replication-v2 + +| Cycle | Protocol | Outcome | Research minutes | Consequential outcome/hour | +| --- | --- | --- | ---: | ---: | +| LEARN-0019 | [protocol](../../research/evolution/cycles/2026-10-04-presentation-replication.md) | inconclusive | unknown | unknown | + +One bounded question counts at most one consequential outcome. Corroboration and inconclusive work remain visible separately. Fewer than three declared cycles: qualitative review trigger pending; no trend assessment. + ## Coherence and maintainability Profile hashes and local references passed validation when this report was generated. That checks traceability, not source truth or external URL freshness. @@ -687,3 +696,17 @@ Evidence: [StorageReceiverRecovery](../../research/reviews/StorageReceiverRecove Remaining uncertainty: Power-loss, device-loss backup, remote durability and arbitrary operation-ID retention remain unverified. No full frontend contract, independent human validation or M1/M2/M3 acceptance. New runtime group is not comparable with static studies; no learning trend. Supersedes: none recorded. + +### LEARN-0019 — Does named organization change constraint/recovery recommendations versus identical guidance under numbered headings on fresh authored variants? + +2026-10-04 · stream cross-stream · prospective · inconclusive + +Before: Consumed pilot-01 model adjudication scored 18/24 in each condition; naming benefit and equivalence remain unestablished. + +Conclusion: The preregistered 12-attempt run produced no retained answers. All six pairs are excluded; no quality comparison or naming conclusion is possible. Endpoint health checks failed inside and outside the sandbox. + +Evidence: [execution-summary](../../research/evaluations/language-replication-02/execution-summary.json), [availability-review](../../research/evaluations/language-replication-02/availability-review.json), [README](../../research/evaluations/language-replication-02/README.md). + +Remaining uncertainty: No usable generation or empirical rating; failed attempts are not zero scores. Provider usage and billing are unknown; no retries or additional calls authorized. Active research effort unknown; request elapsed time measures failed execution only. Independent human validation and real consumer outcomes remain absent; this new group cannot close a comparable three-cycle window. + +Supersedes: none recorded. diff --git a/research/evolution/cycles/2026-10-04-presentation-replication.md b/research/evolution/cycles/2026-10-04-presentation-replication.md index 6123183..0d4a26d 100644 --- a/research/evolution/cycles/2026-10-04-presentation-replication.md +++ b/research/evolution/cycles/2026-10-04-presentation-replication.md @@ -37,3 +37,8 @@ Outcome is recorded after the authorized run in the replication review and LEARN ledger entry. WP-0009 T01 retains independent human pilot-01 ratings; T03 retains M1 re-assessment. WP-0008 retains actual consumer decision/outcome evidence. A new comparison group does not close WP-0006's comparable-window requirement. + + +Recorded outcome: all twelve attempts failed with no answers; six pairs excluded. +See [execution record](../../evaluations/language-replication-02/README.md). +LEARN-0019 is inconclusive; no empirical comparison or quality change follows. diff --git a/research/evolution/ledger.json b/research/evolution/ledger.json index 40cf322..9fcb0d0 100644 --- a/research/evolution/ledger.json +++ b/research/evolution/ledger.json @@ -636,6 +636,42 @@ "cost_usd": null }, "supersedes": [] + }, + { + "id": "LEARN-0019", + "date": "2026-10-04", + "recording": "prospective", + "stream": "cross-stream", + "comparison_group": "presentation-atomic-replication-v2", + "protocol_ref": "research/evolution/cycles/2026-10-04-presentation-replication.md", + "question": "Does named organization change constraint/recovery recommendations versus identical guidance under numbered headings on fresh authored variants?", + "previous_belief": "Consumed pilot-01 model adjudication scored 18/24 in each condition; naming benefit and equivalence remain unestablished.", + "conclusion": "The preregistered 12-attempt run produced no retained answers. All six pairs are excluded; no quality comparison or naming conclusion is possible. Endpoint health checks failed inside and outside the sandbox.", + "disposition": "inconclusive", + "affected_patterns": [], + "evidence": [ + "research/evaluations/language-replication-02/execution-summary.json", + "research/evaluations/language-replication-02/availability-review.json", + "research/evaluations/language-replication-02/README.md" + ], + "remaining_uncertainty": [ + "No usable generation or empirical rating; failed attempts are not zero scores.", + "Provider usage and billing are unknown; no retries or additional calls authorized.", + "Active research effort unknown; request elapsed time measures failed execution only.", + "Independent human validation and real consumer outcomes remain absent; this new group cannot close a comparable three-cycle window." + ], + "reviewer": { + "name": "Codex", + "role": "author-model", + "independent_human": false + }, + "effort": { + "research_minutes": null, + "execution_minutes": 0.000983, + "reported_tokens": null, + "cost_usd": null + }, + "supersedes": [] } ] } diff --git a/research/evolution/snapshots/review-2026-10-04-replication-failure.json b/research/evolution/snapshots/review-2026-10-04-replication-failure.json new file mode 100644 index 0000000..0842515 --- /dev/null +++ b/research/evolution/snapshots/review-2026-10-04-replication-failure.json @@ -0,0 +1,1567 @@ +{ + "schema_version": 1, + "id": "review-2026-10-04-replication-failure", + "source_revision": "1e2741a95909341441da05f86a4b6721eae29b60", + "assessment": { + "profiles": { + "schema_version": 1, + "assessed_at": "2026-09-28", + "profiles": [ + { + "id": "CAND-0001", + "path": "research/candidates/ConflictCopy.md", + "title": "Conflict Copy", + "sha256": "24f250d89ac17d96df287d67c275b5320b48c8356b57ae126a6c941e92a1c6e9", + "status": "draft", + "maturity": "hypothesis", + "decision": "Preserve competing versions instead of silently losing one.", + "evidence": [ + { + "kind": "documentary", + "family": "Joplin", + "ref": "research/reviews/LocalFirstEvidenceExpansion.md", + "scope": "Documented detected-note conflict copy." + }, + { + "kind": "documentary", + "family": "Obsidian", + "ref": "research/reviews/LocalFirstEvidenceExpansion.md", + "scope": "Optional conflict-file mode, not default behavior." + } + ], + "independent_full_contract_uses": null, + "review": { + "assessed_at": "2026-09-27", + "reviewer": "Codex", + "role": "author-model", + "independent_human": false, + "last_substantive_review_date": "2026-09-27", + "substantive_review_ref": "research/reviews/M1-SeedReview.md", + "substantive_review_scope": "Editorial review of the connected core; M1 revise/defer." + }, + "gaps": [ + "Authorized retention and export are separate decisions.", + "Complete contract recurrence and fault-tested deployed behavior remain unverified." + ], + "superseded_by": [] + }, + { + "id": "CAND-0002", + "path": "research/candidates/LocalCommitBoundary.md", + "title": "Local Commit Boundary", + "sha256": "22165fc2979e8f4d8859b34417cb71e7b5ec7a8440ad045aee2383d499648013", + "status": "draft", + "maturity": "hypothesis", + "decision": "Define when a local change is durably accepted.", + "evidence": [ + { + "kind": "documentary", + "family": "Ink & Switch", + "ref": "research/reviews/LocalFirstEvidenceExpansion.md", + "scope": "Local-first motivation." + }, + { + "kind": "documentary", + "family": "Joplin", + "ref": "research/reviews/LocalFirstEvidenceExpansion.md", + "scope": "Local storage and synchronization architecture." + }, + { + "kind": "runtime", + "family": "authored SQLite / Git recovery fixture", + "ref": "research/reviews/StorageReceiverRecovery.md", + "scope": "SQLite WAL/FULL: pre-commit process kill rolls back; post-commit kill preserves document and outbox. No power-loss/backup proof." + } + ], + "independent_full_contract_uses": null, + "review": { + "assessed_at": "2026-09-28", + "reviewer": "Codex", + "role": "author-model", + "independent_human": false, + "last_substantive_review_date": "2026-09-28", + "substantive_review_ref": "research/reviews/StorageReceiverRecovery.md", + "substantive_review_scope": "SQLite WAL/FULL: pre-commit process kill rolls back; post-commit kill preserves document and outbox. No power-loss/backup proof." + }, + "gaps": [ + "Process-kill recovery observed in SQLite fixture; power-loss, device-loss backup and full frontend contract remain unverified.", + "Local acceptance is not off-device recovery." + ], + "superseded_by": [] + }, + { + "id": "CAND-0003", + "path": "research/candidates/PendingSynchronization.md", + "title": "Pending Synchronization", + "sha256": "9c5cb0f4ea4e6f4406ce55513ca2c252172427a49d8280ae1a9c713f1d11b070", + "status": "draft", + "maturity": "hypothesis", + "decision": "Keep accepted local intent visible until synchronization resolves.", + "evidence": [ + { + "kind": "documentary", + "family": "Joplin", + "ref": "research/reviews/LocalFirstEvidenceExpansion.md", + "scope": "Per-item synchronization metadata." + }, + { + "kind": "runtime", + "family": "authored SQLite / Git recovery fixture", + "ref": "research/reviews/StorageReceiverRecovery.md", + "scope": "SQLite atomic supersession/restart plus Git identity replay after lost local acknowledgement; one authored integration, no frontend recurrence." + } + ], + "independent_full_contract_uses": null, + "review": { + "assessed_at": "2026-09-28", + "reviewer": "Codex", + "role": "author-model", + "independent_human": false, + "last_substantive_review_date": "2026-09-28", + "substantive_review_ref": "research/reviews/StorageReceiverRecovery.md", + "substantive_review_scope": "SQLite atomic supersession/restart plus Git identity replay after lost local acknowledgement; one authored integration, no frontend recurrence." + }, + "gaps": [ + "Atomic supersession observed in SQLite fixture; concurrent in-flight supersession, UI states and general receiver retention remain unverified." + ], + "superseded_by": [] + }, + { + "id": "CAND-0004", + "path": "research/candidates/ExplicitReconciliation.md", + "title": "Explicit Reconciliation", + "sha256": "d48a40aab7417b8b8f8406094c84b9e90d0ca8c2f94d6e616d0b80499dc04959", + "status": "draft", + "maturity": "hypothesis", + "decision": "Resolve divergent content while retaining provenance.", + "evidence": [ + { + "kind": "documentary", + "family": "Joplin", + "ref": "research/reviews/LocalFirstEvidenceExpansion.md", + "scope": "Documented manual conflict resolution." + }, + { + "kind": "documentary", + "family": "Obsidian", + "ref": "research/reviews/LocalFirstEvidenceExpansion.md", + "scope": "Conflict comparison workflow." + } + ], + "independent_full_contract_uses": null, + "review": { + "assessed_at": "2026-09-27", + "reviewer": "Codex", + "role": "author-model", + "independent_human": false, + "last_substantive_review_date": "2026-09-27", + "substantive_review_ref": "research/reviews/M1-SeedReview.md", + "substantive_review_scope": "Editorial review of the connected core; M1 revise/defer." + }, + "gaps": [ + "Atomic resolution/provenance UI not executed.", + "Overlap with Conflict Copy remains an open merge question." + ], + "superseded_by": [] + }, + { + "id": "CAND-0005", + "path": "research/candidates/LocalDraftState.md", + "title": "Local Draft State", + "sha256": "8e5f427975c76083c80b895bc68b7130b249be5129c2854f3df11cc374b4b240", + "status": "draft", + "maturity": "hypothesis", + "decision": "Separate unfinished intent from refreshed authoritative data.", + "evidence": [ + { + "kind": "documentary", + "family": "Kubernetes / GitLab / React Router", + "ref": "research/reviews/TransactionalEvidenceExpansion.md", + "scope": "Rejection and pending-input mechanisms motivate recovery." + }, + { + "kind": "static-code", + "family": "Zulip", + "ref": "research/reviews/FrontendRecoveryImplementations.md", + "scope": "Pinned composer draft storage; partial lifecycle." + }, + { + "kind": "static-code", + "family": "Element / Matrix", + "ref": "research/reviews/FrontendRecoveryImplementations.md", + "scope": "Pinned composer draft restoration; partial lifecycle." + }, + { + "kind": "runtime", + "family": "authored release fixture", + "ref": "research/reviews/ActivationConflictsAndReload.md", + "scope": "Same-tab preservation/refusal/discard and post-navigation schema check." + } + ], + "independent_full_contract_uses": null, + "review": { + "assessed_at": "2026-09-27", + "reviewer": "Codex", + "role": "author-model", + "independent_human": false, + "last_substantive_review_date": "2026-09-27", + "substantive_review_ref": "research/reviews/M1-SeedReview.md", + "substantive_review_scope": "Editorial review of the connected core; M1 revise/defer." + }, + "gaps": [ + "No native-unload, crash, multiple-tab or access-revocation guarantee.", + "Full base/draft/current merge contract is not verified in deployed frontends." + ], + "superseded_by": [] + }, + { + "id": "CAND-0006", + "path": "research/candidates/VersionCheckedMutation.md", + "title": "Version-Checked Mutation", + "sha256": "6a3335ae2fa1de51c95dcbfa8d766d988a3f7a9604fb608b8a0bcf9a10b64198", + "status": "draft", + "maturity": "hypothesis", + "decision": "Reject an operation whose observed base is stale.", + "evidence": [ + { + "kind": "documentary", + "family": "Kubernetes", + "ref": "research/reviews/TransactionalEvidenceExpansion.md", + "scope": "Conditional PUT; precise version mechanism." + }, + { + "kind": "documentary", + "family": "GitLab", + "ref": "research/reviews/TransactionalEvidenceExpansion.md", + "scope": "Optional last_commit_id; different failure surface." + }, + { + "kind": "runtime", + "family": "authored release fixture", + "ref": "research/reviews/ActivationConflictsAndReload.md", + "scope": "Related conditional activation-map mechanism in one process, not the full frontend pattern." + }, + { + "kind": "runtime", + "family": "authored SQLite / Git recovery fixture", + "ref": "research/reviews/StorageReceiverRecovery.md", + "scope": "Actual Git receive-pack rejects a forced divergent branch update; related ancestry mechanism, not exact-version frontend semantics." + } + ], + "independent_full_contract_uses": null, + "review": { + "assessed_at": "2026-09-28", + "reviewer": "Codex", + "role": "author-model", + "independent_human": false, + "last_substantive_review_date": "2026-09-28", + "substantive_review_ref": "research/reviews/StorageReceiverRecovery.md", + "substantive_review_scope": "Actual Git receive-pack rejects a forced divergent branch update; related ancestry mechanism, not exact-version frontend semantics." + }, + "gaps": [ + "Distributed atomicity and complete frontend recurrence remain unverified.", + "Refetch does not authorize overwriting conflicting intent." + ], + "superseded_by": [] + }, + { + "id": "CAND-0007", + "path": "research/candidates/MutationOutcomeTracking.md", + "title": "Mutation Outcome Tracking", + "sha256": "e04cb835fd054bfaf88616375bd0262deffb2801ea04105be92543bfa9582179", + "status": "draft", + "maturity": "hypothesis", + "decision": "Retain operation identity when transport outcome is uncertain.", + "evidence": [ + { + "kind": "documentary", + "family": "Stripe", + "ref": "research/reviews/TransactionalEvidenceExpansion.md", + "scope": "Idempotency mechanism." + }, + { + "kind": "documentary", + "family": "React Router", + "ref": "research/reviews/TransactionalEvidenceExpansion.md", + "scope": "Cancellation does not imply server rollback." + }, + { + "kind": "static-code", + "family": "Zulip", + "ref": "research/reviews/FrontendRecoveryImplementations.md", + "scope": "Pinned send/recovery state." + }, + { + "kind": "static-code", + "family": "Element / Matrix", + "ref": "research/reviews/FrontendRecoveryImplementations.md", + "scope": "Pinned client and SDK transaction/echo paths." + }, + { + "kind": "runtime", + "family": "Element / Matrix", + "ref": "research/reviews/MatrixRecoveryExperiment.md", + "scope": "Real SDK with synthetic transport: lost acknowledgement, identity reuse and authoritative echo." + }, + { + "kind": "runtime", + "family": "authored SQLite / Git recovery fixture", + "ref": "research/reviews/StorageReceiverRecovery.md", + "scope": "Actual Git same-commit replay is a no-op, reachable identity survives GC, fresh identity repeats intent; no arbitrary operation ledger." + } + ], + "independent_full_contract_uses": null, + "review": { + "assessed_at": "2026-09-28", + "reviewer": "Codex", + "role": "author-model", + "independent_human": false, + "last_substantive_review_date": "2026-09-28", + "substantive_review_ref": "research/reviews/StorageReceiverRecovery.md", + "substantive_review_scope": "Actual Git same-commit replay is a no-op, reachable identity survives GC, fresh identity repeats intent; no arbitrary operation ledger." + }, + "gaps": [ + "No production server deduplication or complete retention-window guarantee.", + "Cancelling local UI is not remote compensation." + ], + "superseded_by": [] + }, + { + "id": "CAND-0008", + "path": "research/candidates/InvalidateAndReproject.md", + "title": "Invalidate and Reproject", + "sha256": "47c405adbc3caac2383768ed3d30b6a2e7ff1f44f4954c340049e78944227844", + "status": "draft", + "maturity": "hypothesis", + "decision": "Refresh a projection after mutation without equating acceptance and visibility.", + "evidence": [ + { + "kind": "documentary", + "family": "React Router", + "ref": "research/reviews/TransactionalEvidenceExpansion.md", + "scope": "Concurrency and pending-state guidance." + } + ], + "independent_full_contract_uses": null, + "review": { + "assessed_at": "2026-09-27", + "reviewer": "Codex", + "role": "author-model", + "independent_human": false, + "last_substantive_review_date": "2026-09-27", + "substantive_review_ref": "research/reviews/M1-SeedReview.md", + "substantive_review_scope": "Editorial review of the connected core; M1 revise/defer." + }, + "gaps": [ + "Read-your-write and acceptance/refresh distinction still proposed.", + "No independent deployed full-contract observation." + ], + "superseded_by": [] + }, + { + "id": "CAND-0009", + "path": "research/candidates/URLAddressableState.md", + "title": "URL-addressable State", + "sha256": "07d524c3c0dd080bd42ff5dd79a36e6ecd2a8654f07ece880b483477f7ca4497", + "status": "draft", + "maturity": "hypothesis", + "decision": "Make a navigable state reconstructible from its URL.", + "evidence": [ + { + "kind": "documentary", + "family": "htmx", + "ref": "research/reviews/NavigationRenderingEvidence.md", + "scope": "History behavior and history-cache misses." + }, + { + "kind": "documentary", + "family": "Web platform", + "ref": "research/reviews/NavigationRenderingEvidence.md", + "scope": "pushState changes history, not server dereferenceability." + }, + { + "kind": "runtime", + "family": "htmx in authored public-catalog fixture", + "ref": "research/reviews/NavigationHistoryExperiment.md", + "scope": "NH01–NH06: native/enhanced routes, history hit/miss, actual browser cache ordering and missing-Vary negative control; explicit Node validator checks separate." + }, + { + "kind": "runtime", + "family": "htmx in authored public-catalog fixture", + "ref": "research/reviews/NavigationOwnershipExperiment.md", + "scope": "NO01–NO06: ordered history/success/error races, prior policy 0/12 full obligations versus guarded 12/12; separate focus ownership. Same fixture family; pinned history-XHR adapter." + }, + { + "kind": "static-code", + "family": "Zulip", + "ref": "research/reviews/ZulipNavigationSourceStudy.md", + "scope": "ZN01–ZN08 at pinned revision: partial hash/list/cache mechanisms and non-modal continuation contrast. No application/test execution, no no-script claim, no complete pattern recurrence." + }, + { + "kind": "static-code", + "family": "Zulip", + "ref": "research/reviews/ZulipTargetLookupCallPath.md", + "scope": "ZT01-ZT04: traced target-message-lookup and anchor-navigation continuations to their unconditional completion points; no captured-list guard or test coverage located. No reproduction or application execution." + } + ], + "independent_full_contract_uses": null, + "review": { + "assessed_at": "2026-09-27", + "reviewer": "Codex", + "role": "author-model", + "independent_human": false, + "last_substantive_review_date": "2026-09-27", + "substantive_review_ref": "research/reviews/ZulipTargetLookupCallPath.md", + "substantive_review_scope": "Traced call-path corroboration of the previously flagged auxiliary-callback gap; still author-model, no application execution." + }, + "gaps": [ + "Independent application recurrence and human/AT validation remain unverified.", + "Pending history disclosure, multiple history misses, redirects, nonzero swap delays, bfcache, lifecycle cleanup, interrupted/late enhancement and offline availability remain open.", + "Zulip's target-lookup and anchor-navigation continuations have a traced absence of a captured-current-list guard and no located test coverage for a stale-navigation race (unreproduced, not a defect claim); deferred editing focus and search callback wiring remain separately unverified." + ], + "superseded_by": [] + }, + { + "id": "CAND-0010", + "path": "research/candidates/ProgressiveEnhancement.md", + "title": "Progressive Enhancement", + "sha256": "1dc3885ef896b9a7cb61af11fa8893484eb1e13454207326abc7250d5ca4382b", + "status": "draft", + "maturity": "hypothesis", + "decision": "Keep essential actions usable when enhancement is unavailable.", + "evidence": [ + { + "kind": "documentary", + "family": "htmx", + "ref": "research/reviews/NavigationRenderingEvidence.md", + "scope": "Boosted links and forms." + }, + { + "kind": "documentary", + "family": "SvelteKit", + "ref": "research/reviews/NavigationRenderingEvidence.md", + "scope": "Form actions and rendering/page options." + }, + { + "kind": "runtime", + "family": "htmx in authored public-catalog fixture", + "ref": "research/reviews/NavigationHistoryExperiment.md", + "scope": "NH01–NH06: native/enhanced routes, history hit/miss, actual browser cache ordering and missing-Vary negative control; explicit Node validator checks separate." + }, + { + "kind": "runtime", + "family": "htmx in authored public-catalog fixture", + "ref": "research/reviews/NavigationFocusExperiment.md", + "scope": "NF01–NF06: Tab/Enter navigation, raw versus authored focus/status policy, held-response ownership and background update; DOM evidence only, not AT speech." + }, + { + "kind": "runtime", + "family": "htmx in authored public-catalog fixture", + "ref": "research/reviews/NavigationOwnershipExperiment.md", + "scope": "NO01–NO06: ordered history/success/error races, prior policy 0/12 full obligations versus guarded 12/12; separate focus ownership. Same fixture family; pinned history-XHR adapter." + }, + { + "kind": "static-code", + "family": "Zulip", + "ref": "research/reviews/ZulipNavigationSourceStudy.md", + "scope": "ZN01–ZN08 at pinned revision: partial hash/list/cache mechanisms and non-modal continuation contrast. No application/test execution, no no-script claim, no complete pattern recurrence." + }, + { + "kind": "static-code", + "family": "Zulip", + "ref": "research/reviews/ZulipTargetLookupCallPath.md", + "scope": "ZT01-ZT04: traced target-message-lookup and anchor-navigation continuations to their unconditional completion points; no captured-list guard or test coverage located. No reproduction or application execution." + } + ], + "independent_full_contract_uses": null, + "review": { + "assessed_at": "2026-09-27", + "reviewer": "Codex", + "role": "author-model", + "independent_human": false, + "last_substantive_review_date": "2026-09-27", + "substantive_review_ref": "research/reviews/ZulipTargetLookupCallPath.md", + "substantive_review_scope": "Traced call-path corroboration of the previously flagged auxiliary-callback gap; still author-model, no application execution." + }, + "gaps": [ + "Independent application recurrence and human/AT validation remain unverified.", + "Pending history disclosure, multiple history misses, redirects, nonzero swap delays, bfcache, lifecycle cleanup, interrupted/late enhancement and offline availability remain open.", + "Zulip's target-lookup and anchor-navigation continuations have a traced absence of a captured-current-list guard and no located test coverage for a stale-navigation race (unreproduced, not a defect claim); deferred editing focus and search callback wiring remain separately unverified." + ], + "superseded_by": [] + }, + { + "id": "CAND-0011", + "path": "research/candidates/PresentationModel.md", + "title": "Presentation Model", + "sha256": "5ff03b5799b0a9e5ec6b54b21034720d89dbc8ffde7c4cdcf71616cdf57baf49", + "status": "draft", + "maturity": "hypothesis", + "decision": "Separate presentation decisions from view mechanisms.", + "evidence": [ + { + "kind": "documentary", + "family": "Fowler", + "ref": "research/reviews/PatternBreadthExpansion.md", + "scope": "Named pattern and illustrative model/view discussion." + } + ], + "independent_full_contract_uses": null, + "review": { + "assessed_at": "2026-09-27", + "reviewer": "Codex", + "role": "author-model", + "independent_human": false, + "last_substantive_review_date": "2026-09-26", + "substantive_review_ref": "research/reviews/PatternBreadthExpansion.md", + "substantive_review_scope": "Source normalization, decision probes and counterexamples." + }, + "gaps": [ + "No executed model/view integration or independently reviewed full-contract use." + ], + "superseded_by": [] + }, + { + "id": "CAND-0012", + "path": "research/candidates/DerivedState.md", + "title": "Derived State", + "sha256": "9f29b9f1296a52b83f08dfd97437c4dc16e4949ddac1ff3b3830e3fc9831bd4f", + "status": "draft", + "maturity": "hypothesis", + "decision": "Compute a result from explicit authoritative inputs.", + "evidence": [ + { + "kind": "documentary", + "family": "Redux", + "ref": "research/reviews/PatternBreadthExpansion.md", + "scope": "Selector guidance, not a requirement for a global store." + } + ], + "independent_full_contract_uses": null, + "review": { + "assessed_at": "2026-09-27", + "reviewer": "Codex", + "role": "author-model", + "independent_human": false, + "last_substantive_review_date": "2026-09-26", + "substantive_review_ref": "research/reviews/PatternBreadthExpansion.md", + "substantive_review_scope": "Source normalization, decision probes and counterexamples." + }, + "gaps": [ + "Snapshot coherence and changed-locale cache test still planned." + ], + "superseded_by": [] + }, + { + "id": "CAND-0013", + "path": "research/candidates/VerticalFeatureSlice.md", + "title": "Vertical Feature Slice", + "sha256": "d6b332b9c104a079bd4b72512b8ddd1a889199b2e2e531753c411688053e647a", + "status": "draft", + "maturity": "hypothesis", + "decision": "Group a capability across UI, state and behavior with explicit interfaces.", + "evidence": [ + { + "kind": "documentary", + "family": "Feature-Sliced Design", + "ref": "research/reviews/PatternBreadthExpansion.md", + "scope": "Layer/slice and public-interface guidance." + }, + { + "kind": "static-code", + "family": "Feature-Sliced Design", + "ref": "research/reviews/FeatureDeploymentBoundaries.md", + "scope": "Pinned todo-app interfaces coexist with one application build." + } + ], + "independent_full_contract_uses": null, + "review": { + "assessed_at": "2026-09-27", + "reviewer": "Codex", + "role": "author-model", + "independent_human": false, + "last_substantive_review_date": "2026-09-26", + "substantive_review_ref": "research/reviews/FeatureDeploymentBoundaries.md", + "substantive_review_scope": "Static feature/import versus release-boundary assessment." + }, + "gaps": [ + "Build/import enforcement not executed.", + "Feature organization does not establish independent activation." + ], + "superseded_by": [] + }, + { + "id": "CAND-0014", + "path": "research/candidates/IslandsOfInteractivity.md", + "title": "Islands of Interactivity", + "sha256": "821a1497da10c02ae11fa6f56cb8e205a4f32c40b39770ab8ca3f907561a260a", + "status": "draft", + "maturity": "hypothesis", + "decision": "Activate selected interactive regions within a document.", + "evidence": [ + { + "kind": "documentary", + "family": "Astro", + "ref": "research/reviews/PatternBreadthExpansion.md", + "scope": "Client-island mechanism and execution placement." + } + ], + "independent_full_contract_uses": null, + "review": { + "assessed_at": "2026-09-27", + "reviewer": "Codex", + "role": "author-model", + "independent_human": false, + "last_substantive_review_date": "2026-09-26", + "substantive_review_ref": "research/reviews/PatternBreadthExpansion.md", + "substantive_review_scope": "Source normalization, decision probes and counterexamples." + }, + "gaps": [ + "No executed island fixture, performance comparison or complete recurrence study." + ], + "superseded_by": [] + }, + { + "id": "CAND-0015", + "path": "research/candidates/FailureBoundary.md", + "title": "Failure Boundary", + "sha256": "a6e76ed17a92baade683d5df96fa8ecd066c593661e6a14983a86d046e7d9c2c", + "status": "draft", + "maturity": "hypothesis", + "decision": "Contain specified failure classes while preserving useful neighboring work.", + "evidence": [ + { + "kind": "documentary", + "family": "React", + "ref": "research/reviews/PatternBreadthExpansion.md", + "scope": "Error boundary scope and exclusions." + }, + { + "kind": "runtime", + "family": "React", + "ref": "research/reviews/BoundaryRecoveryExperiment.md", + "scope": "Real package in authored host: render/event/request distinction and fallback escalation." + } + ], + "independent_full_contract_uses": null, + "review": { + "assessed_at": "2026-09-27", + "reviewer": "Codex", + "role": "author-model", + "independent_human": false, + "last_substantive_review_date": "2026-09-26", + "substantive_review_ref": "research/reviews/BoundaryRecoveryExperiment.md", + "substantive_review_scope": "Authored runtime fixture and failure-class interpretation." + }, + "gaps": [ + "Synthetic host, not full application recurrence.", + "Boundary cannot imply containment for all asynchronous failures." + ], + "superseded_by": [] + }, + { + "id": "CAND-0016", + "path": "research/candidates/FocusReturn.md", + "title": "Focus Return", + "sha256": "9149c2f563eafcfec983a9285949225840d97804568309bd16bbb1eb6b3dde77", + "status": "draft", + "maturity": "hypothesis", + "decision": "Restore meaningful focus after a temporary interaction ends.", + "evidence": [ + { + "kind": "documentary", + "family": "WAI-ARIA APG", + "ref": "research/reviews/PatternBreadthExpansion.md", + "scope": "Dialog focus obligations." + }, + { + "kind": "static-code", + "family": "Radix", + "ref": "research/reviews/NestedDismissalIsolation.md", + "scope": "Pinned layer/listener ownership implementation." + }, + { + "kind": "runtime", + "family": "Radix", + "ref": "research/reviews/NestedOwnershipRouting.md", + "scope": "Authored fixture: raw 20/24 versus routed 24/24; browser/build limits retained." + }, + { + "kind": "runtime", + "family": "htmx in authored public-catalog fixture", + "ref": "research/reviews/NavigationFocusExperiment.md", + "scope": "NF01–NF06: Tab/Enter navigation, raw versus authored focus/status policy, held-response ownership and background update; DOM evidence only, not AT speech." + }, + { + "kind": "runtime", + "family": "htmx in authored public-catalog fixture", + "ref": "research/reviews/NavigationOwnershipExperiment.md", + "scope": "NO01–NO06: ordered history/success/error races, prior policy 0/12 full obligations versus guarded 12/12; separate focus ownership. Same fixture family; pinned history-XHR adapter." + }, + { + "kind": "static-code", + "family": "Zulip", + "ref": "research/reviews/ZulipNavigationSourceStudy.md", + "scope": "ZN01–ZN08 at pinned revision: partial hash/list/cache mechanisms and non-modal continuation contrast. No application/test execution, no no-script claim, no complete pattern recurrence." + } + ], + "independent_full_contract_uses": null, + "review": { + "assessed_at": "2026-09-27", + "reviewer": "Codex", + "role": "author-model", + "independent_human": false, + "last_substantive_review_date": "2026-09-27", + "substantive_review_ref": "research/reviews/ZulipNavigationSourceStudy.md", + "substantive_review_scope": "Static navigation contrast; partial mechanisms, not independent full-contract use." + }, + "gaps": [ + "One implementation family; repeated trials are not independent uses.", + "Animated exits and competing owners remain untested; no production repair claim.", + "Route ownership is a contrast, not independent modal recurrence; AT speech and broader history/lifecycle cases remain unverified.", + "Zulip auxiliary target/anchor callbacks, deferred editing focus and search callback wiring remain unverified; static guards do not prove whole-application ownership." + ], + "superseded_by": [] + }, + { + "id": "CAND-0017", + "path": "research/candidates/IndependentFeatureBoundary.md", + "title": "Independent Feature Boundary", + "sha256": "60b78725ed4036681675d4c68244fc25aea6e0dffb09d54a7108e3fcbf527b4f", + "status": "draft", + "maturity": "hypothesis", + "decision": "Separate feature release, activation and session transition contracts.", + "evidence": [ + { + "kind": "documentary", + "family": "single-spa / webpack", + "ref": "research/reviews/FeatureDeploymentBoundaries.md", + "scope": "Mechanism documentation, not deployed independence proof." + }, + { + "kind": "static-code", + "family": "react-microfrontends examples", + "ref": "research/reviews/FeatureDeploymentBoundaries.md", + "scope": "Root config/navbar are one example family with shared startup gating." + }, + { + "kind": "runtime", + "family": "authored release fixture", + "ref": "research/reviews/FeatureReleaseExperiment.md", + "scope": "Publication/activation/rollback across fresh, loaded and lazy sessions." + }, + { + "kind": "runtime", + "family": "authored release fixture", + "ref": "research/reviews/ActivationConflictsAndReload.md", + "scope": "Conditional activation updates and safe reload policies, including negative controls." + } + ], + "independent_full_contract_uses": null, + "review": { + "assessed_at": "2026-09-27", + "reviewer": "Codex", + "role": "author-model", + "independent_human": false, + "last_substantive_review_date": "2026-09-26", + "substantive_review_ref": "research/reviews/ActivationConflictsAndReload.md", + "substantive_review_scope": "Authored activation/reload experiment and guarantee limits." + }, + "gaps": [ + "No upstream deployment or real team/release-independence validation.", + "Distributed atomicity, retention authorization and native-unload/crash behavior remain open." + ], + "superseded_by": [] + }, + { + "id": "CAND-0018", + "path": "research/candidates/CapabilityContractBoundary.md", + "title": "Capability Contract Boundary", + "sha256": "a678f88e9b53d7273e29e3aea2d23f2297055253dcf4518444d3089fa6da4331", + "status": "draft", + "maturity": "hypothesis", + "decision": "Declare a UI capability's semantic surface separately from its runtime-isolation mechanism.", + "evidence": [ + { + "kind": "documentary", + "family": "Web Components", + "ref": "research/reviews/CapabilityContractSources.md", + "scope": "CC01: Custom Elements observedAttributes/lifecycle callback mechanism, narrower than a full semantic contract." + }, + { + "kind": "documentary", + "family": "webpack", + "ref": "research/reviews/CapabilityContractSources.md", + "scope": "CC02: Module Federation expose/consume module and share-scope dependency-negotiation mechanism, one layer below any UI-facing contract." + } + ], + "independent_full_contract_uses": null, + "review": { + "assessed_at": "2026-09-27", + "reviewer": "Codex", + "role": "author-model", + "independent_human": false, + "last_substantive_review_date": "2026-09-27", + "substantive_review_ref": "research/reviews/CapabilityContractSources.md", + "substantive_review_scope": "Two real substrate specs read directly; the prompting AI-chat transcript is not evidence." + }, + "gaps": [ + "No known use of an explicit capability contract, as distinct from its substrate mechanisms, has been located.", + "Enforcement, versioning and compatibility checking are unverified; a contract with no tooling is only documentation.", + "Only one family per substrate mechanism inspected; no independent recurrence." + ], + "superseded_by": [] + } + ] + }, + "ledger": { + "schema_version": 1, + "coverage": "Ten selected retrospective outcomes (not a historical census), followed by prospectively registered bounded cycles. Retrospective effort remains unknown.", + "events": [ + { + "id": "LEARN-0001", + "date": "2026-09-25", + "recording": "retrospective", + "stream": "D", + "comparison_group": null, + "question": "What do pinned recovery implementations establish?", + "previous_belief": "Documentary motivation did not locate draft/outcome mechanisms in frontend code.", + "conclusion": "Zulip and Element locate partial draft/outcome mechanisms; complete pattern recurrence is still unknown.", + "disposition": "revision", + "affected_patterns": [ + "CAND-0005", + "CAND-0007" + ], + "evidence": [ + "research/reviews/FrontendRecoveryImplementations.md" + ], + "remaining_uncertainty": [ + "Deployed full lifecycle remains unverified." + ], + "reviewer": { + "name": "Codex", + "role": "author-model", + "independent_human": false + }, + "effort": { + "research_minutes": null, + "execution_minutes": null, + "reported_tokens": null, + "cost_usd": null + }, + "supersedes": [] + }, + { + "id": "LEARN-0002", + "date": "2026-09-25", + "recording": "retrospective", + "stream": "D", + "comparison_group": null, + "question": "Does cancellation or a transport error settle mutation outcome?", + "previous_belief": "Recovery identity and authoritative echo required executable checks.", + "conclusion": "Synthetic transport with the real Matrix SDK distinguishes replay identity, duplicate commits and authoritative echo.", + "disposition": "revision", + "affected_patterns": [ + "CAND-0007" + ], + "evidence": [ + "research/reviews/MatrixRecoveryExperiment.md" + ], + "remaining_uncertainty": [ + "No real server deduplication guarantee." + ], + "reviewer": { + "name": "Codex", + "role": "author-model", + "independent_human": false + }, + "effort": { + "research_minutes": null, + "execution_minutes": null, + "reported_tokens": null, + "cost_usd": null + }, + "supersedes": [] + }, + { + "id": "LEARN-0003", + "date": "2026-09-26", + "recording": "retrospective", + "stream": "cross-stream", + "comparison_group": null, + "question": "Does naming patterns improve the frozen pilot decisions?", + "previous_belief": "Initial editorial scores were 21/24 named and 19/24 prose.", + "conclusion": "Submitted model review adjudication yields 18/24 in each condition; this does not establish equivalence or a presentation benefit.", + "disposition": "revision", + "affected_patterns": [ + "CAND-0005", + "CAND-0007", + "CAND-0001", + "CAND-0002" + ], + "evidence": [ + "research/evaluations/language-pilot-01/submitted-review-01/adjudication.md" + ], + "remaining_uncertainty": [ + "Independent human validation pending.", + "Historical effort unknown; this entry does not allocate whole-pilot telemetry." + ], + "reviewer": { + "name": "Codex", + "role": "author-model", + "independent_human": false + }, + "effort": { + "research_minutes": null, + "execution_minutes": null, + "reported_tokens": null, + "cost_usd": null + }, + "supersedes": [] + }, + { + "id": "LEARN-0004", + "date": "2026-09-26", + "recording": "retrospective", + "stream": "D", + "comparison_group": null, + "question": "Which recovery recommendations overstate their guarantees?", + "previous_belief": "Shared guidance left retention/export policy, cancellation and local backup boundaries insufficiently explicit.", + "conclusion": "Guidance now separates authorization, remote compensation, atomic supersession and off-device backup.", + "disposition": "revision", + "affected_patterns": [ + "CAND-0001", + "CAND-0002", + "CAND-0003", + "CAND-0005", + "CAND-0007" + ], + "evidence": [ + "research/reviews/SharedGuidanceRevision.md" + ], + "remaining_uncertainty": [ + "Editorial correction is not new production evidence." + ], + "reviewer": { + "name": "Codex", + "role": "author-model", + "independent_human": false + }, + "effort": { + "research_minutes": null, + "execution_minutes": null, + "reported_tokens": null, + "cost_usd": null + }, + "supersedes": [] + }, + { + "id": "LEARN-0005", + "date": "2026-09-26", + "recording": "retrospective", + "stream": "B", + "comparison_group": null, + "question": "What does URL/history and enhancement documentation actually promise?", + "previous_belief": "Navigation and rendering seeds lacked explicit dereferenceability and fallback distinctions.", + "conclusion": "Developed two hypotheses separating address-bar updates from direct loads, and HTML delivery from usable no-script actions.", + "disposition": "revision", + "affected_patterns": [ + "CAND-0009", + "CAND-0010" + ], + "evidence": [ + "research/reviews/NavigationRenderingEvidence.md" + ], + "remaining_uncertainty": [ + "Navigation/history/cache runtime checks remain pending." + ], + "reviewer": { + "name": "Codex", + "role": "author-model", + "independent_human": false + }, + "effort": { + "research_minutes": null, + "execution_minutes": null, + "reported_tokens": null, + "cost_usd": null + }, + "supersedes": [] + }, + { + "id": "LEARN-0006", + "date": "2026-09-26", + "recording": "retrospective", + "stream": "F", + "comparison_group": null, + "question": "Why can immediate nested dismissal close the outer task?", + "previous_belief": "The first fixture exposed a timing failure with unresolved ownership.", + "conclusion": "Trace locates stale Escape listener ownership despite inner registration/focus; a guard can lose input.", + "disposition": "revision", + "affected_patterns": [ + "CAND-0016", + "CAND-0015" + ], + "evidence": [ + "research/reviews/BoundaryRecoveryExperiment.md", + "research/reviews/NestedDismissalIsolation.md" + ], + "remaining_uncertainty": [ + "Readiness waits are not a public product repair." + ], + "reviewer": { + "name": "Codex", + "role": "author-model", + "independent_human": false + }, + "effort": { + "research_minutes": null, + "execution_minutes": null, + "reported_tokens": null, + "cost_usd": null + }, + "supersedes": [] + }, + { + "id": "LEARN-0007", + "date": "2026-09-26", + "recording": "retrospective", + "stream": "F", + "comparison_group": null, + "question": "Can event-time ownership route dismissal without a settling delay?", + "previous_belief": "Isolation supplied a mechanism but no lifecycle-safe candidate routing experiment.", + "conclusion": "Controlled fixture routed 24/24 versus raw 20/24; initial measurement error and corrected results retained.", + "disposition": "revision", + "affected_patterns": [ + "CAND-0016" + ], + "evidence": [ + "research/reviews/NestedOwnershipRouting.md" + ], + "remaining_uncertainty": [ + "Animated exit and competing-owner coverage remain open." + ], + "reviewer": { + "name": "Codex", + "role": "author-model", + "independent_human": false + }, + "effort": { + "research_minutes": null, + "execution_minutes": null, + "reported_tokens": null, + "cost_usd": null + }, + "supersedes": [] + }, + { + "id": "LEARN-0008", + "date": "2026-09-26", + "recording": "retrospective", + "stream": "E", + "comparison_group": null, + "question": "Are feature organization and independent activation the same boundary?", + "previous_belief": "The feature-slice hypothesis needed a source-layout/release contrast.", + "conclusion": "Pinned examples distinguish organizational slicing from activation contracts; added Independent Feature Boundary.", + "disposition": "revision", + "affected_patterns": [ + "CAND-0013", + "CAND-0017" + ], + "evidence": [ + "research/reviews/FeatureDeploymentBoundaries.md" + ], + "remaining_uncertainty": [ + "Shared startup dependencies still couple nominally separate features." + ], + "reviewer": { + "name": "Codex", + "role": "author-model", + "independent_human": false + }, + "effort": { + "research_minutes": null, + "execution_minutes": null, + "reported_tokens": null, + "cost_usd": null + }, + "supersedes": [] + }, + { + "id": "LEARN-0009", + "date": "2026-09-26", + "recording": "retrospective", + "stream": "E", + "comparison_group": null, + "question": "Does publication or rollback update every session?", + "previous_belief": "Static source review could not establish loaded/lazy-session behavior.", + "conclusion": "Native-module experiment distinguishes publication, activation and already-bound sessions; artifact retirement breaks old lazy sessions.", + "disposition": "revision", + "affected_patterns": [ + "CAND-0017" + ], + "evidence": [ + "research/reviews/FeatureReleaseExperiment.md" + ], + "remaining_uncertainty": [ + "No CDN or upstream deployment validation." + ], + "reviewer": { + "name": "Codex", + "role": "author-model", + "independent_human": false + }, + "effort": { + "research_minutes": null, + "execution_minutes": null, + "reported_tokens": null, + "cost_usd": null + }, + "supersedes": [] + }, + { + "id": "LEARN-0010", + "date": "2026-09-26", + "recording": "retrospective", + "stream": "E", + "comparison_group": null, + "question": "How do stale activation writers and unfinished drafts constrain updates?", + "previous_belief": "Release experiment left activation conflicts and reload policy unexecuted.", + "conclusion": "Conditional replacement rejects stale writers; explicit reload policies preserve or refuse transition, with schema recheck after navigation.", + "disposition": "revision", + "affected_patterns": [ + "CAND-0017", + "CAND-0005", + "CAND-0006" + ], + "evidence": [ + "research/reviews/ActivationConflictsAndReload.md" + ], + "remaining_uncertainty": [ + "One-process atomicity; no native unload, crash or real remote-outcome coverage." + ], + "reviewer": { + "name": "Codex", + "role": "author-model", + "independent_human": false + }, + "effort": { + "research_minutes": null, + "execution_minutes": null, + "reported_tokens": null, + "cost_usd": null + }, + "supersedes": [] + }, + { + "id": "LEARN-0011", + "date": "2026-09-27", + "recording": "prospective", + "stream": "B", + "comparison_group": "B-navigation-mechanism-v1", + "protocol_ref": "research/evolution/cycles/2026-09-27-navigation.md", + "question": "Can a public route preserve selection and complete-document availability across native/enhanced entry, history and HTTP caching?", + "previous_belief": "Native href/GET contracts plus representation-aware routing and cache selection should satisfy this narrow obligation; documentary support lacked runtime checks.", + "conclusion": "Corroborated in pinned htmx fixture: native and history paths work; missing Vary reproduces fragment-as-document reuse in both browsers. Existing obligations stand; no new pattern or promotion.", + "disposition": "corroboration", + "affected_patterns": [ + "CAND-0009", + "CAND-0010" + ], + "evidence": [ + "research/reviews/NavigationHistoryExperiment.md", + "research/experiments/navigation-history/result.json", + "research/evolution/cycles/2026-09-27-navigation-effort.json" + ], + "remaining_uncertainty": [ + "One authored application, no independent recurrence or usefulness gain.", + "Accessibility, delayed scripts, interrupted transport, real intermediaries and offline availability remain open." + ], + "reviewer": { + "name": "Codex", + "role": "author-model", + "independent_human": false + }, + "effort": { + "research_minutes": 6.177, + "execution_minutes": 1.021, + "reported_tokens": null, + "cost_usd": null + }, + "supersedes": [] + }, + { + "id": "LEARN-0012", + "date": "2026-09-27", + "recording": "prospective", + "stream": "F", + "comparison_group": "F-navigation-focus-v1", + "protocol_ref": "research/evolution/cycles/2026-09-27-navigation-focus.md", + "question": "Does enhanced route replacement need explicit continuation and feedback policy that respects changed focus ownership?", + "previous_belief": "Native controls alone do not establish enhanced focus/feedback semantics; delayed responses should respect focus moved to another surviving task. Modal return is not automatically route continuation.", + "conclusion": "Corroborated in 22 cases: raw same-ID link focus survives but this fragment contract leaves stale title and silent failure; authored heading/status policy works and preserves newer search focus. No AT speech or universal heading policy established.", + "disposition": "corroboration", + "affected_patterns": [ + "CAND-0010", + "CAND-0016" + ], + "evidence": [ + "research/reviews/NavigationFocusExperiment.md", + "research/experiments/navigation-history/focus-result.json", + "research/evolution/cycles/2026-09-27-navigation-focus-effort.json" + ], + "remaining_uncertainty": [ + "Author-model DOM evidence, no human/AT validation or independent recurrence.", + "Delayed history focus, competing requests, delayed scripts and interrupted enhancement remain open." + ], + "reviewer": { + "name": "Codex", + "role": "author-model", + "independent_human": false + }, + "effort": { + "research_minutes": 5.138, + "execution_minutes": 0.525, + "reported_tokens": null, + "cost_usd": null + }, + "supersedes": [] + }, + { + "id": "LEARN-0013", + "date": "2026-09-27", + "recording": "prospective", + "stream": "F", + "comparison_group": "F-navigation-ownership-v1", + "protocol_ref": "research/evolution/cycles/2026-09-27-navigation-ownership.md", + "question": "Can obsolete history/navigation completions affect newer intent, and does separating completion ownership from focus ownership prevent it?", + "previous_belief": "The single-response continuation policy preserved newer search focus but left competing history/success/error completions unresolved; focus-at-swap alone might not be sufficient.", + "conclusion": "Revise operational guidance: validate navigation ownership before content/URL/title/feedback effects, including stale errors and newer failure, independently of focus permission. Prior policy misses obligations in 12/12 adversarial rows; authored guard meets 12/12. Pause fixture work for independent source contrast.", + "disposition": "revision", + "affected_patterns": [ + "CAND-0009", + "CAND-0010", + "CAND-0016" + ], + "evidence": [ + "research/reviews/NavigationOwnershipExperiment.md", + "research/reviews/NavigationThreadReview.md", + "research/experiments/navigation-history/ownership-result.json", + "research/evolution/cycles/2026-09-27-navigation-ownership-effort.json" + ], + "remaining_uncertainty": [ + "One authored htmx family and author-model review, not production recurrence or AT validation.", + "History adapter is pinned implementation-dependent; pending-state, multiple-miss and lifecycle limits remain open.", + "Three distinct protocols are not a comparable diminishing-returns window." + ], + "reviewer": { + "name": "Codex", + "role": "author-model", + "independent_human": false + }, + "effort": { + "research_minutes": 13.918, + "execution_minutes": 1.238, + "reported_tokens": null, + "cost_usd": null + }, + "supersedes": [] + }, + { + "id": "LEARN-0014", + "date": "2026-09-27", + "recording": "prospective", + "stream": "B", + "comparison_group": "B-independent-navigation-source-v1", + "protocol_ref": "research/evolution/cycles/2026-09-27-zulip-navigation.md", + "question": "Where does independently developed Zulip scope navigation/search result and focus applicability, and where does our model fail to establish a guarantee?", + "previous_belief": "The authored navigation generation guard protected current-view and focus effects, but independent application recurrence and alternative ownership/cache policies were unverified.", + "conclusion": "Revise guidance toward per-effect applicability: pinned Zulip preserves noncurrent cache data while guarding main selected-view continuations by list identity. Make client-fragment addressability explicit without inferring no-script enhancement. Shared feedback, auxiliary callbacks and deferred focus limit the inference.", + "disposition": "revision", + "affected_patterns": [ + "CAND-0009", + "CAND-0010", + "CAND-0016" + ], + "evidence": [ + "research/reviews/ZulipNavigationSourceStudy.md", + "research/reviews/ZulipNavigationSources.json", + "research/evolution/cycles/2026-09-27-zulip-navigation-effort.json" + ], + "remaining_uncertainty": [ + "Static targeted ranges in twelve files, no Zulip runtime/upstream test execution or independent human/AT review.", + "Auxiliary lookup/anchor continuations, deferred focus and callback wiring need a bounded follow-up before any race claim.", + "One Zulip family supplies partial mechanisms, not full-contract recurrence or a no-script guarantee." + ], + "reviewer": { + "name": "Codex", + "role": "author-model", + "independent_human": false + }, + "effort": { + "research_minutes": 6.345, + "execution_minutes": 0.289, + "reported_tokens": null, + "cost_usd": null + }, + "supersedes": [] + }, + { + "id": "LEARN-0015", + "date": "2026-09-27", + "recording": "prospective", + "stream": "cross-stream", + "comparison_group": "CD-authority-projection-documentary-v1", + "protocol_ref": "research/evolution/cycles/2026-09-27-authority-projection.md", + "question": "Do the six authority/projection seeds denote reusable patterns or a mixture of scoped acceptance, projection placement and delivery choices?", + "previous_belief": "Authority is scoped and acknowledgement levels differ, but six seed names lack developed contracts and their projection/delivery relationships are not explicitly compared.", + "conclusion": "Revise the research decision aid to separate scoped acceptance, projection computation and delivery timing with compatibility constraints. Six seed dispositions and two synthetic traces; retain all seeds, zero new drafts or candidate maturity changes.", + "disposition": "revision", + "affected_patterns": [], + "evidence": [ + "research/reviews/AuthorityProjectionComparison.md", + "research/evolution/cycles/2026-09-27-authority-projection-effort.json" + ], + "remaining_uncertainty": [ + "Six living primary documents across four families, not independent full-contract implementations; exact revisions unknown.", + "Synthetic VC-01/02 reasoning is not execution or demonstrated usefulness; authority/merge/storage and phase-specific recovery contracts still need application evidence.", + "Existing candidate profiles unchanged; five prospective groups are not a comparable learning trend." + ], + "reviewer": { + "name": "Codex", + "role": "author-model", + "independent_human": false + }, + "effort": { + "research_minutes": 3.128, + "execution_minutes": 0.033, + "reported_tokens": null, + "cost_usd": null + }, + "supersedes": [] + }, + { + "id": "LEARN-0016", + "date": "2026-09-27", + "recording": "prospective", + "stream": "cross-stream", + "comparison_group": "CD-acceptance-projection-static-v1", + "protocol_ref": "research/evolution/cycles/2026-09-27-zulip-acceptance.md", + "question": "Does the six-question authority/projection aid distinguish local editable content, remote shared acceptance and subsequent projection in a pinned application?", + "previous_belief": "Scoped acceptance, projection location and delivery schedule are separate questions with compatibility constraints; a local display need not establish shared acceptance.", + "conclusion": "Corroborated in pinned Zulip content editing: conditional local Markdown echo precedes remote validation; locked optional content checks and server rendering precede save/event scheduling; HTTP and event application are distinct paths. Same displayed object can carry tentative content. No candidate or maturity change.", + "disposition": "corroboration", + "affected_patterns": [], + "evidence": [ + "research/reviews/ZulipAcceptanceProjection.md", + "research/reviews/ZulipAcceptanceSources.json", + "research/evolution/cycles/2026-09-27-zulip-acceptance-effort.json" + ], + "remaining_uncertainty": [ + "Ten selected source/test files from one existing Zulip family, no application/database/upstream test execution or full-contract recurrence.", + "Response/event ordering, lost response after acceptance and current-edit ownership need preregistered reproduction; no defect or safe-replay claim.", + "Six prospective groups are not a comparable trend; original human and milestone gates remain open." + ], + "reviewer": { + "name": "Codex", + "role": "author-model", + "independent_human": false + }, + "effort": { + "research_minutes": 4.182, + "execution_minutes": 0.102, + "reported_tokens": null, + "cost_usd": null + }, + "supersedes": [] + }, + { + "id": "LEARN-0017", + "date": "2026-09-27", + "recording": "prospective", + "stream": "B", + "comparison_group": "B-independent-navigation-source-v1", + "protocol_ref": "research/evolution/cycles/2026-09-27-zulip-target-lookup.md", + "question": "Do Zulip's target-message lookup and anchor-navigation fetch continuation guard against a stale navigation that occurred while the request was in flight, the way the main narrow-fetch continuation (ZN02) does?", + "previous_belief": "ZN07 located both callbacks and noted no visible captured-current-list guard, but did not trace their callers or check test coverage.", + "conclusion": "Traced both continuations to their concrete completion points: process_result calls opts.cont unconditionally, and neither the target-lookup channel.get nor load_messages_around_anchor supplies opts.msg_list, so neither participates in the opt-in list-identity guard that message_fetch.ts implements for other callers. No located test exercises a stale-navigation scenario for either path.", + "disposition": "corroboration", + "affected_patterns": [ + "CAND-0009", + "CAND-0010" + ], + "evidence": [ + "research/reviews/ZulipTargetLookupCallPath.md", + "research/reviews/ZulipTargetLookupSources.json", + "research/evolution/cycles/2026-09-27-zulip-target-lookup-effort.json" + ], + "remaining_uncertainty": [ + "Static source trace only; no application, database or upstream test executed, and no race actually reproduced.", + "show() performs its own state reset on entry, so a stale re-entry's externally observable effect is unverified without execution.", + "Second cycle in comparison group B-independent-navigation-source-v1; still short of three comparable cycles, and the workplan's rule against manufacturing cycles to reach that count still applies." + ], + "reviewer": { + "name": "Codex", + "role": "author-model", + "independent_human": false + }, + "effort": { + "research_minutes": null, + "execution_minutes": null, + "reported_tokens": null, + "cost_usd": null + }, + "supersedes": [] + }, + { + "id": "LEARN-0018", + "date": "2026-09-28", + "recording": "prospective", + "stream": "D", + "comparison_group": "D-sqlite-git-recovery-v1", + "protocol_ref": "research/evolution/cycles/2026-09-28-storage-receiver.md", + "question": "Which process-recovery obligations can SQLite plus actual Git receive-pack satisfy, and where does object identity fail to supply an operation receipt?", + "previous_belief": "Atomic local intent/outbox updates and stable receiver identity support recovery within explicit transaction, ancestry and retention boundaries.", + "conclusion": "Nine checks corroborate atomic SQLite process recovery, Git replay and server ancestry refusal; split-transaction and fresh-identity negative controls retain the limits. One authored integration, not a full frontend use.", + "disposition": "corroboration", + "affected_patterns": [ + "CAND-0002", + "CAND-0003", + "CAND-0006", + "CAND-0007" + ], + "evidence": [ + "research/reviews/StorageReceiverRecovery.md", + "research/experiments/storage-receiver/result.json", + "research/evolution/cycles/2026-09-28-storage-receiver-effort.json" + ], + "remaining_uncertainty": [ + "Power-loss, device-loss backup, remote durability and arbitrary operation-ID retention remain unverified.", + "No full frontend contract, independent human validation or M1/M2/M3 acceptance.", + "New runtime group is not comparable with static studies; no learning trend." + ], + "reviewer": { + "name": "Codex", + "role": "author-model", + "independent_human": false + }, + "effort": { + "research_minutes": 4.123, + "execution_minutes": 0.014, + "reported_tokens": null, + "cost_usd": null + }, + "supersedes": [] + }, + { + "id": "LEARN-0019", + "date": "2026-10-04", + "recording": "prospective", + "stream": "cross-stream", + "comparison_group": "presentation-atomic-replication-v2", + "protocol_ref": "research/evolution/cycles/2026-10-04-presentation-replication.md", + "question": "Does named organization change constraint/recovery recommendations versus identical guidance under numbered headings on fresh authored variants?", + "previous_belief": "Consumed pilot-01 model adjudication scored 18/24 in each condition; naming benefit and equivalence remain unestablished.", + "conclusion": "The preregistered 12-attempt run produced no retained answers. All six pairs are excluded; no quality comparison or naming conclusion is possible. Endpoint health checks failed inside and outside the sandbox.", + "disposition": "inconclusive", + "affected_patterns": [], + "evidence": [ + "research/evaluations/language-replication-02/execution-summary.json", + "research/evaluations/language-replication-02/availability-review.json", + "research/evaluations/language-replication-02/README.md" + ], + "remaining_uncertainty": [ + "No usable generation or empirical rating; failed attempts are not zero scores.", + "Provider usage and billing are unknown; no retries or additional calls authorized.", + "Active research effort unknown; request elapsed time measures failed execution only.", + "Independent human validation and real consumer outcomes remain absent; this new group cannot close a comparable three-cycle window." + ], + "reviewer": { + "name": "Codex", + "role": "author-model", + "independent_human": false + }, + "effort": { + "research_minutes": null, + "execution_minutes": 0.000983, + "reported_tokens": null, + "cost_usd": null + }, + "supersedes": [] + } + ] + }, + "context": { + "schema_version": 1, + "cases": [ + { + "id": "VC-01", + "state": "synthetic draft", + "ref": "research/walkthroughs/VC01-TransactionalAdministration.md" + }, + { + "id": "VC-02", + "state": "synthetic draft", + "ref": "research/walkthroughs/VC02-OfflineFieldNotes.md" + }, + { + "id": "VC-03", + "state": "provisional", + "ref": "research/walkthroughs/VC03-ContentPublication.md" + }, + { + "id": "VC-04", + "state": "planned", + "ref": "research/plan/ValidationCases.md" + }, + { + "id": "VC-05", + "state": "planned", + "ref": "research/plan/ValidationCases.md" + }, + { + "id": "VC-06", + "state": "provisional", + "ref": "research/walkthroughs/VC06-MicrofrontendEstate.md" + }, + { + "id": "VC-07", + "state": "planned", + "ref": "research/plan/ValidationCases.md" + }, + { + "id": "VC-08", + "state": "planned", + "ref": "research/plan/ValidationCases.md" + }, + { + "id": "VC-09", + "state": "planned", + "ref": "research/plan/ValidationCases.md" + }, + { + "id": "VC-10", + "state": "planned", + "ref": "research/plan/ValidationCases.md" + } + ], + "usefulness": { + "named_score": 18, + "prose_score": 18, + "denominator": 24, + "independent_human": false, + "decision_time_minutes": null, + "ref": "research/evaluations/language-pilot-01/submitted-review-01/adjudication.md", + "limit": "One frozen development pilot with submitted model review; six matched briefs. Equal aggregate scores do not establish equivalence. Stable anchors are consumed development cases, not fresh held-out tasks.", + "initial_unsupported_claims_each": 2, + "adjudicated_unsupported_claim_count": null + }, + "open_work": [ + "workplans/FEP-WP-0009-m1-independent-validation.md", + "workplans/FEP-WP-0003-contrasting-schools.md", + "workplans/FEP-WP-0006-language-evolution.md", + "workplans/FEP-WP-0008-use-driven-improvement.md" + ] + } + } +} diff --git a/research/reviews/M1-SeedReview.md b/research/reviews/M1-SeedReview.md index e47f9a4..febe8f0 100644 --- a/research/reviews/M1-SeedReview.md +++ b/research/reviews/M1-SeedReview.md @@ -65,3 +65,12 @@ No candidate promotion, ID rename or evidence-profile upgrade occurs. These are explicit work records, not a claim that the remaining work was completed by packaging the reference. The frozen pilot and original investigation receipts are unchanged. Review this gate again when those tasks deliver new evidence. + + +## Fresh replication attempt — 2026-10-04 + +[Replication 02](../evaluations/language-replication-02/README.md) implements six +fresh authored variants, equivalent guidance, atomic criteria and a frozen bounded +runner. All twelve authorized attempts failed without answers; all pairs are excluded. +No empirical scores or independent human evidence were added. M1 stays **revise/defer**; +WP-0009 T01/T02/T03 retain independent review, usable replication and reassessment. diff --git a/tests/test_language_replication.py b/tests/test_language_replication.py index 4a28d35..3861286 100644 --- a/tests/test_language_replication.py +++ b/tests/test_language_replication.py @@ -56,7 +56,10 @@ def test_tampered_prompt_guidance_or_blank_sheet_is_rejected(packet, path): def test_no_calls_before_frozen_inputs_are_committed(packet): - with pytest.raises(subprocess.CalledProcessError): + # Before packet publication Git may report a missing path; after publication + # the fresh temporary assignment/freeze differs from HEAD. Both must stop + # execution before adapter construction. + with pytest.raises((subprocess.CalledProcessError, ValueError)): replication.run(12, packet, lambda: pytest.fail('No adapter before commit verification')) diff --git a/workplans/FEP-WP-0006-language-evolution.md b/workplans/FEP-WP-0006-language-evolution.md index bfb8cf1..ec5dc08 100644 --- a/workplans/FEP-WP-0006-language-evolution.md +++ b/workplans/FEP-WP-0006-language-evolution.md @@ -253,3 +253,14 @@ local links/whitespace passed. [Review snapshot](../research/evolution/snapshots pins the committed assessment at implementation revision `d8d3a22`; it is a measurement-review boundary, not an additional empirical cycle. The original baseline and all previous snapshots are unchanged. + + +## Retained failed replication cycle — 2026-10-04 + +LEARN-0019 preregistered at `1e2741a` is an inconclusive attempted presentation +replication in a new protocol group. All twelve attempts failed; no usable answers, +ratings or quality result. Research effort is unknown; request duration is separately +measured. The ledger now has nine prospective attempts/cycles in eight groups, +including this wholly failed run. Largest group remains two, with LEARN-0017 effort +unknown. This failure is retained and cannot establish returns or close T03's +three-cycle prerequisite. WP-0009 T02 owns restoration/future authorized replication. diff --git a/workplans/FEP-WP-0009-m1-independent-validation.md b/workplans/FEP-WP-0009-m1-independent-validation.md index 7ea85ca..b070437 100644 --- a/workplans/FEP-WP-0009-m1-independent-validation.md +++ b/workplans/FEP-WP-0009-m1-independent-validation.md @@ -4,7 +4,7 @@ type: workplan title: "Obtain independent validation for the M1 seed gate" domain: infotech repo: frontend-patterns -status: active +status: blocked owner: codex topic_slug: canon origin: residual @@ -43,7 +43,8 @@ submitted-review-01 worksheet was a model review and does not count. ```task id: FEP-WP-0009-T02 -status: progress +status: wait +blocking_reason: "All 12 authorized attempts failed; llm-connect endpoint unreachable. A future replication requires restored service and fresh explicit execution authorization; no retries allowed." priority: medium state_hub_task_id: "8e0ac763-eba4-5625-ad63-af53322bcac4" ``` @@ -94,3 +95,19 @@ The user then explicitly authorized **up to 12 calls, 1,400 output tokens per ca no retries**. This is the fresh replication budget only, not a grant for further model reviews, autonomous editing, policy changes or human-score substitution. Freeze and commit the packet/runner before execution; reserve every attempt and retain failures. + +## Retained failed replication — 2026-10-04 + +The preregistered packet/runner was committed and synced at `1e2741a` before +execution. All twelve reserved attempts failed with no retained answer or provider +usage receipt. Both sandbox and host health probes found the configured llm-connect +endpoint unreachable. No call was retried. The budget is conservatively exhausted +as twelve reserved attempts, without claiming twelve provider executions or zero cost. + +[Execution record](../research/evaluations/language-replication-02/README.md) retains +all failures, null ratings and six excluded pairs. Packet construction, validation +and bounded execution are implemented; T02 remains wait because no empirical +comparison exists. Future execution needs a new registered packet/run identity and +fresh explicit authorization after service restoration. T01 and T03 retain their +existing gates. LEARN-0019 records one inconclusive attempted cycle, not useful +validation or independent evidence.