Preregister fresh atomic-criteria replication with 12-call budget
Assistant: codex Assistant-Model: gpt-6.1-sol Assistant-Session: 01a1045b-933d-7aa1-b2db-104557487a79
This commit is contained in:
parent
bd85fcd179
commit
1e2741a959
30 changed files with 2229 additions and 2 deletions
7
Makefile
7
Makefile
|
|
@ -40,3 +40,10 @@ reference-test:
|
|||
.PHONY: consumption-test
|
||||
consumption-test:
|
||||
PYTHONDONTWRITEBYTECODE=1 "$(BENCH_PY)" -m pytest -q -p no:cacheprovider tests/test_reference_consume.py
|
||||
|
||||
.PHONY: replication-check replication-test
|
||||
replication-check:
|
||||
python3 scripts/language_replication.py check
|
||||
|
||||
replication-test:
|
||||
PYTHONDONTWRITEBYTECODE=1 "$(BENCH_PY)" -m pytest -q -p no:cacheprovider tests/test_language_replication.py
|
||||
|
|
|
|||
14
docs/evidence/2026-10-04-consumer-discovery.json
Normal file
14
docs/evidence/2026-10-04-consumer-discovery.json
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
{
|
||||
"observed_at": "2026-10-04T08:54:07.636647+00:00",
|
||||
"method": "State Hub todo/progress tasks joined to active/ready workplans; title plus first 200 description characters matched bounded frontend/interaction vocabulary. Coordination index, not independent source eligibility or global absence proof.",
|
||||
"examined_tasks_by_status": {
|
||||
"todo": 49,
|
||||
"progress": 2
|
||||
},
|
||||
"candidates": [],
|
||||
"limitations": [
|
||||
"Only active/ready plans and indexed task text considered.",
|
||||
"A candidate requires source/task inspection and owner integration before enrollment.",
|
||||
"No consumer task created, reopened or enrolled by this discovery."
|
||||
]
|
||||
}
|
||||
44
research/evaluations/language-replication-02/README.md
Normal file
44
research/evaluations/language-replication-02/README.md
Normal file
|
|
@ -0,0 +1,44 @@
|
|||
# Fresh presentation replication 02
|
||||
|
||||
WP-0009 T02 implementation: six new authored variants, six atomic criteria per
|
||||
scenario, two equivalent guidance presentations and twelve isolated prompts.
|
||||
The prior consumed pilot remains separate and unchanged. This replication is not
|
||||
held-out M3 retrieval, a real consumer use or independent human validation.
|
||||
|
||||
The [protocol](protocol.md) defines interpretation and budgets;
|
||||
[scenarios](scenarios.json) define the frozen obligations and prohibited claims.
|
||||
`pack-equivalence.json` records shared body/source hashes. `freeze.json` pins all
|
||||
inputs, prompts, assignment and code plus immutable reference version/corpus/package.
|
||||
The public assignment supports auditing but makes future masking procedural.
|
||||
|
||||
Commands from the repository root:
|
||||
|
||||
```bash
|
||||
make replication-check replication-test
|
||||
```
|
||||
|
||||
The user authorized up to twelve llm-connect calls at a requested 1,400-output-token
|
||||
cap per call with no retries. Execution occurs only after a committed freeze:
|
||||
|
||||
```bash
|
||||
PYTHONDONTWRITEBYTECODE=1 ../infospace-bench/.venv/bin/python scripts/language_replication.py run --max-calls 12
|
||||
```
|
||||
|
||||
The runner reserves attempts before requests. A second invocation skips every reserved
|
||||
session and can call only unstarted sessions within the aggregate ceiling; it never
|
||||
retries a failed/interrupted attempt. Raw receipts preserve usage, model identity,
|
||||
output hashes and failed responses. Scoring reads `masked/` before opening the key.
|
||||
Copy the empty `scores-template.json`, disclose reviewer role and prior exposure,
|
||||
record binary scores with quotes/locators and reasons, and list unsupported claims
|
||||
separately. Null means unavailable or unreviewed; it is never a default zero.
|
||||
|
||||
```bash
|
||||
python3 scripts/language_replication.py scores-check --scores <review-file.json> --complete
|
||||
python3 scripts/language_replication.py summarize --scores <review-file.json>
|
||||
```
|
||||
|
||||
Validation checks structure and retained quotations/hashes; it does not assess semantic
|
||||
correctness or independently authenticate a declared reviewer. Keep initial ratings,
|
||||
disagreements and adjudication. A failed pair is excluded from quality counts, and no
|
||||
score automatically completes human review or M1. Actual execution/editorial outcomes
|
||||
are recorded here after the authorized run; independent-human scores remain pending.
|
||||
74
research/evaluations/language-replication-02/assignment.json
Normal file
74
research/evaluations/language-replication-02/assignment.json
Normal file
|
|
@ -0,0 +1,74 @@
|
|||
[
|
||||
{
|
||||
"run": "r01",
|
||||
"scenario": "R02-S01",
|
||||
"condition": "patterns",
|
||||
"blind_id": "R02-S01-U"
|
||||
},
|
||||
{
|
||||
"run": "r02",
|
||||
"scenario": "R02-S01",
|
||||
"condition": "prose",
|
||||
"blind_id": "R02-S01-V"
|
||||
},
|
||||
{
|
||||
"run": "r03",
|
||||
"scenario": "R02-S02",
|
||||
"condition": "prose",
|
||||
"blind_id": "R02-S02-V"
|
||||
},
|
||||
{
|
||||
"run": "r04",
|
||||
"scenario": "R02-S02",
|
||||
"condition": "patterns",
|
||||
"blind_id": "R02-S02-U"
|
||||
},
|
||||
{
|
||||
"run": "r05",
|
||||
"scenario": "R02-S03",
|
||||
"condition": "patterns",
|
||||
"blind_id": "R02-S03-U"
|
||||
},
|
||||
{
|
||||
"run": "r06",
|
||||
"scenario": "R02-S03",
|
||||
"condition": "prose",
|
||||
"blind_id": "R02-S03-V"
|
||||
},
|
||||
{
|
||||
"run": "r07",
|
||||
"scenario": "R02-S04",
|
||||
"condition": "prose",
|
||||
"blind_id": "R02-S04-U"
|
||||
},
|
||||
{
|
||||
"run": "r08",
|
||||
"scenario": "R02-S04",
|
||||
"condition": "patterns",
|
||||
"blind_id": "R02-S04-V"
|
||||
},
|
||||
{
|
||||
"run": "r09",
|
||||
"scenario": "R02-S05",
|
||||
"condition": "patterns",
|
||||
"blind_id": "R02-S05-U"
|
||||
},
|
||||
{
|
||||
"run": "r10",
|
||||
"scenario": "R02-S05",
|
||||
"condition": "prose",
|
||||
"blind_id": "R02-S05-V"
|
||||
},
|
||||
{
|
||||
"run": "r11",
|
||||
"scenario": "R02-S06",
|
||||
"condition": "prose",
|
||||
"blind_id": "R02-S06-U"
|
||||
},
|
||||
{
|
||||
"run": "r12",
|
||||
"scenario": "R02-S06",
|
||||
"condition": "patterns",
|
||||
"blind_id": "R02-S06-V"
|
||||
}
|
||||
]
|
||||
45
research/evaluations/language-replication-02/freeze.json
Normal file
45
research/evaluations/language-replication-02/freeze.json
Normal file
|
|
@ -0,0 +1,45 @@
|
|||
{
|
||||
"schema_version": 1,
|
||||
"packet": "language-replication-02",
|
||||
"prepared_at": "2026-10-04T09:05:26.684061+00:00",
|
||||
"reference": {
|
||||
"packaging_revision": "53bb65e7a6eb1a2fa933e5b41dfa5a0e7409d14d",
|
||||
"manifest_sha256": "d7f60ac04817820b805265d1d747cfbb8d2269b7bd6244cd9d3e8d2fd44ef0b8",
|
||||
"version": "0.1",
|
||||
"corpus_revision": "8cc7016e1fbcb08b670091a174672fcdd8d07640"
|
||||
},
|
||||
"files": {
|
||||
"research/evaluations/language-replication-02/protocol.md": "9bfcdfbab869ff979cad8e259a017e911ef6053a78f6f18b00de152a43d4d12a",
|
||||
"research/evaluations/language-replication-02/instructions.txt": "3b809a8b48d9be525e306f895f9230884f016c3a5c218adbecdeffb9fa2c43e0",
|
||||
"research/evaluations/language-replication-02/scenarios.json": "f32345310e2fa2c93301729eedca9332bb98a897a029453f974909bbe3554cdd",
|
||||
"research/evaluations/language-replication-02/guidance.json": "d86f7a6ad8d9941e5cf6683136aa1428d1fb6bf6704f86f94cb760b3b8124d2f",
|
||||
"research/evaluations/language-replication-02/pack-patterns.md": "b0e02e52bf6e968bc011fef97296982e16790d66798864fb39940d001be45edd",
|
||||
"research/evaluations/language-replication-02/pack-prose.md": "e92f26bfe1109b4c633e9a8daaf05bc4eeff548778cdbc34dbe40876887fd9b5",
|
||||
"research/evaluations/language-replication-02/pack-equivalence.json": "31afc32ac3f400a22787f39de6902cffa5ac1b7bf39621090ff9c1f44050927a",
|
||||
"research/evaluations/language-replication-02/assignment.json": "87f5edb0caa9909c922c38e29e9b9c9b8ac214df680e12fcb7ca36fd292a0b41",
|
||||
"research/evaluations/language-replication-02/scores-template.json": "7ce11f340bcf1b2f37df90f6bfc73c87a3207737e00244cfc88d730262837e41",
|
||||
"research/evaluations/language-replication-02/requests/r01.txt": "91a50e9c1bcb85b650ca7c51155d7c8aed3d8a95f0375a3c87fbcc735b10dde4",
|
||||
"research/evaluations/language-replication-02/requests/r02.txt": "146a74fc4e3adafd8039553a3676102132551faa8168a5a8fcb401d2b205272c",
|
||||
"research/evaluations/language-replication-02/requests/r03.txt": "be924ba36625b663e3957b29d30303b8042f5a382cb70b11ea111a2762635dc3",
|
||||
"research/evaluations/language-replication-02/requests/r04.txt": "3371ff49a2d0f29ac8a65a768f72aebf0228fb5bf950494878ec4980c8b1ff70",
|
||||
"research/evaluations/language-replication-02/requests/r05.txt": "7a61c90e25d342750b3d9651c90b157975e441822d66e826b1315bce957cafe3",
|
||||
"research/evaluations/language-replication-02/requests/r06.txt": "76ca0bca70f535da292885af2ead28162daac79e5417b1f92aa68a3d024b7145",
|
||||
"research/evaluations/language-replication-02/requests/r07.txt": "d2e369f6381d6bcc769234a870b079eadec29589308c4ad6a8bc4d17ab035311",
|
||||
"research/evaluations/language-replication-02/requests/r08.txt": "8705d35a4bc045405986796e6dcd96d687a82b409e6d0208e697bea5c5d207d1",
|
||||
"research/evaluations/language-replication-02/requests/r09.txt": "8944e9bd64680ed927ece8c8e1ad9104b589f5e6b21fa7872ec35e5d728d90c1",
|
||||
"research/evaluations/language-replication-02/requests/r10.txt": "734d42b631b58b06246db211270c10823100e5e3249964aad286dd9e725d725c",
|
||||
"research/evaluations/language-replication-02/requests/r11.txt": "c2007173765706f16ed61ff93f1246b960ca31b3cee5b6e511863606f23a6433",
|
||||
"research/evaluations/language-replication-02/requests/r12.txt": "b4c71336ea9dd8563db6432e00f1344ebc3379258364f390af73854df89ab432",
|
||||
"scripts/language_replication.py": "593367314086318970abafaf74692f055203cb94cbc21e25e0f4852eafba787a",
|
||||
"scripts/reference_consume.py": "5ee8fdbe1bcd1f0e3a3449fc228000041d1836c4b0ac80c3080b9266ab2254cc",
|
||||
"scripts/reference_preview.py": "a3a7b5f4698db6b7fda9c518158cc449e0897bbd0644775559218972d4f4c18a",
|
||||
"scripts/classification_coverage.py": "89515b27824563beaf13e999051e50d4c2cc4d01b627fbb237de0628a0bb52cf",
|
||||
"scripts/research_bench.py": "1dade65e456ab7b85d17c2c4c9a6e9ecef245104177a8cad815a685d2c7cf427"
|
||||
},
|
||||
"sessions": 12,
|
||||
"max_output_tokens_per_call": 1400,
|
||||
"limits": [
|
||||
"Author-created fresh variants, not held-out M3 retrieval or independent validation.",
|
||||
"No inference at preparation; execution needs explicit user authorization."
|
||||
]
|
||||
}
|
||||
46
research/evaluations/language-replication-02/guidance.json
Normal file
46
research/evaluations/language-replication-02/guidance.json
Normal file
|
|
@ -0,0 +1,46 @@
|
|||
{
|
||||
"intro": "Research reference: all mechanisms are draft hypotheses with limited scoped evidence. Apply only where their conditions hold; do not infer backend, storage, access, retention or backup guarantees.",
|
||||
"items": [
|
||||
{
|
||||
"candidate": "CAND-0005",
|
||||
"title": "Local Draft State",
|
||||
"body": "Keep the base observation, unfinished proposal and current authoritative observation separate. A refresh, old receipt or rejected attempt must not erase newer typing. Decide navigation/restart lifetimes explicitly. Preservation, continued access, retention and deletion are conditional on access policy; recoverability is not permission to retain restricted content. Export needs a separate permission. A missing policy is unknown, not an unconditional retention or deletion rule."
|
||||
},
|
||||
{
|
||||
"candidate": "CAND-0006",
|
||||
"title": "Version-Checked Mutation",
|
||||
"body": "When a proposal depends on an observed base, require the authority to check the expected version atomically with validation and commit. A stale rejection creates a review decision; do not blindly substitute a new version and resend. Preserve the proposal where policy permits and compare current state. A version precondition prevents certain stale writes; an operation identity and its retention contract address retry duplication. Neither substitutes for the other."
|
||||
},
|
||||
{
|
||||
"candidate": "CAND-0007",
|
||||
"title": "Mutation Outcome Tracking",
|
||||
"body": "Track each operation as pending, accepted, explicitly rejected or unknown using operation-specific evidence. A timeout, cancelled browser wait or generic transport error does not prove rejection. A matching authenticated receipt can establish acceptance; weaker later errors cannot reverse it or clear newer work. Lookup and replay need the original immutable payload/identity, authority scope and documented retention window. Missing identity, expired receipts or ambiguous lookup absence require reconciliation, not a guaranteed fresh-ID retry. Stopping local tracking does not undo acceptance. Remote cancellation has domain preconditions and races; reversal of an accepted effect is a separate authorized operation whose outcome must also be tracked."
|
||||
},
|
||||
{
|
||||
"candidate": "CAND-0008",
|
||||
"title": "Invalidate and Reproject",
|
||||
"body": "After acceptance, refresh affected observations while keeping unfinished work separate. Acceptance and view freshness are different states: a failed refresh leaves an accepted operation with unresolved observations, not a reason to replay the write. Guard obsolete read effects, but local request ordering cannot order opaque server versions or repair a replica consistency contract. Verify read-after-write/server ordering before claiming currentness. An authoritative result or suitable subscription may avoid a separate fetch."
|
||||
},
|
||||
{
|
||||
"candidate": "CAND-0002",
|
||||
"title": "Local Commit Boundary",
|
||||
"body": "For a declared restart-safe offline save, commit the revision and pending propagation intent together under a named recoverable storage contract, then acknowledge saved locally. Storage failure leaves an unacknowledged save. Local acknowledgement is not remote acceptance. Ordinary process restart, interrupted commit, power loss, corruption and device loss need distinct evidence. Untested recovery is unknown rather than impossible. Same-device copies and synchronization do not establish independent backup. Define retention/failure scope and test clean-store restoration, including pending identities and reconciliation, before promising off-device recovery."
|
||||
},
|
||||
{
|
||||
"candidate": "CAND-0003",
|
||||
"title": "Pending Synchronization",
|
||||
"body": "Persist stable revision and operation identities with pending work, distinguishing queued, sent and acknowledged states. Receiver acceptance does not prove every other device has received it or that an independent backup exists. Define deduplication, authorization and acknowledgement scope before replay. When resolution replaces operation P with revision c and operation Q, commit c, Q and P's superseded/non-retryable status in one recoverable transaction. After restart, P must not remain active. Local retirement cannot recall already transmitted P; receiver preconditions or reconciliation remain necessary. A late receipt for P updates only P's history, not Q or c."
|
||||
},
|
||||
{
|
||||
"candidate": "CAND-0001",
|
||||
"title": "Conflict Copy",
|
||||
"body": "When divergence is detected and policy permits preservation, retain alternate content with distinct identity and provenance before destructive replacement. A selected working version is not a completed semantic resolution. Access, retention and export remain separate decisions, including after revocation. Copies may fail through missed detection, inaccessible notification, interrupted preservation or shared device failure; they are not a backup or universal atomicity guarantee. Domain-aware merge, rejection or a version store can be alternatives."
|
||||
},
|
||||
{
|
||||
"candidate": "CAND-0004",
|
||||
"title": "Explicit Reconciliation",
|
||||
"body": "Expose the unresolved versions and provenance, then let an authorized person compare, choose or combine them and commit the new decision. Where retention permits, preserve recovery material until that commit; revocation policy can require earlier deletion and limit the available comparison. Provide perceivable conflict state, keyboard comparison and recoverable focus as explicit requirements. Another device may resolve concurrently, so the new revision still needs divergence handling. A revision store can provide alternatives without separate conflict files."
|
||||
}
|
||||
],
|
||||
"closing": "Connections are conditional: unfinished proposals support recovery after stale rejection; operation evidence informs refreshing observations; recoverable local commits enable resumable propagation; divergence can require preserved alternatives and an explicit resolution producing another local commit. No universal ordering or full-pattern recurrence is established. Documentary, pinned-source and authored-runtime mechanisms do not certify a complete deployed architecture or independent usefulness."
|
||||
}
|
||||
|
|
@ -0,0 +1 @@
|
|||
Review the proposed frontend decision. Use the supplied reference only where its conditions hold. Return at most 220 words in four short bullets: (1) operation and visible state, (2) information and policy boundaries, (3) recovery action and rejected alternative, (4) missing contract. Explain reasoning without naming or quoting reference headings, candidate names, identifiers or presentation style. Do not infer undocumented guarantees. No tools, browsing, scores or tables. This is advice, not a claim that you implemented or tested anything.
|
||||
|
|
@ -0,0 +1,57 @@
|
|||
{
|
||||
"method": "Only headings vary; all eight ordered bodies, introduction and closing are byte-identical.",
|
||||
"blocks": [
|
||||
{
|
||||
"candidate": "CAND-0005",
|
||||
"body_sha256": "3f546323d701978976ea14d5f25df2160e34b9d44ac8a755fbab89bf32d9de27",
|
||||
"source_path": "research/candidates/LocalDraftState.md",
|
||||
"source_sha256": "8e5f427975c76083c80b895bc68b7130b249be5129c2854f3df11cc374b4b240"
|
||||
},
|
||||
{
|
||||
"candidate": "CAND-0006",
|
||||
"body_sha256": "0e78b6591bba05ae77568150eb499f73ba720a9b74ccd40dd29f1b06157a6b8d",
|
||||
"source_path": "research/candidates/VersionCheckedMutation.md",
|
||||
"source_sha256": "6a3335ae2fa1de51c95dcbfa8d766d988a3f7a9604fb608b8a0bcf9a10b64198"
|
||||
},
|
||||
{
|
||||
"candidate": "CAND-0007",
|
||||
"body_sha256": "d78c218be17b0a23058ac4728685dff2835c8a2a6c83c7a7d2b343b2e142e279",
|
||||
"source_path": "research/candidates/MutationOutcomeTracking.md",
|
||||
"source_sha256": "e04cb835fd054bfaf88616375bd0262deffb2801ea04105be92543bfa9582179"
|
||||
},
|
||||
{
|
||||
"candidate": "CAND-0008",
|
||||
"body_sha256": "0ac4c907aff3347476a36e935812e36b9f9ab2eea2d097aa0334dc626bc1f5ba",
|
||||
"source_path": "research/candidates/InvalidateAndReproject.md",
|
||||
"source_sha256": "47c405adbc3caac2383768ed3d30b6a2e7ff1f44f4954c340049e78944227844"
|
||||
},
|
||||
{
|
||||
"candidate": "CAND-0002",
|
||||
"body_sha256": "4bd39b91e56b13db81b0401febd54e96eda438e60d21c4bf6b81860fd5507d74",
|
||||
"source_path": "research/candidates/LocalCommitBoundary.md",
|
||||
"source_sha256": "22165fc2979e8f4d8859b34417cb71e7b5ec7a8440ad045aee2383d499648013"
|
||||
},
|
||||
{
|
||||
"candidate": "CAND-0003",
|
||||
"body_sha256": "5e8d19968a026106d8953dda7b615c381368b35ee76d55a3a4c2bf9bd529d814",
|
||||
"source_path": "research/candidates/PendingSynchronization.md",
|
||||
"source_sha256": "9c5cb0f4ea4e6f4406ce55513ca2c252172427a49d8280ae1a9c713f1d11b070"
|
||||
},
|
||||
{
|
||||
"candidate": "CAND-0001",
|
||||
"body_sha256": "9f78a65637ebc97a6e2cf07ac3d66e8c76940fac86d1251ae2112f6d2ebc8b81",
|
||||
"source_path": "research/candidates/ConflictCopy.md",
|
||||
"source_sha256": "24f250d89ac17d96df287d67c275b5320b48c8356b57ae126a6c941e92a1c6e9"
|
||||
},
|
||||
{
|
||||
"candidate": "CAND-0004",
|
||||
"body_sha256": "ff4d87d33c243c653e9030ba4b3c10d58a76f817cba27aed893d23d4392252ce",
|
||||
"source_path": "research/candidates/ExplicitReconciliation.md",
|
||||
"source_sha256": "d48a40aab7417b8b8f8406094c84b9e90d0ca8c2f94d6e616d0b80499dc04959"
|
||||
}
|
||||
],
|
||||
"word_counts": {
|
||||
"patterns": 741,
|
||||
"prose": 737
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
Research reference: all mechanisms are draft hypotheses with limited scoped evidence. Apply only where their conditions hold; do not infer backend, storage, access, retention or backup guarantees.
|
||||
|
||||
## Local Draft State
|
||||
|
||||
Keep the base observation, unfinished proposal and current authoritative observation separate. A refresh, old receipt or rejected attempt must not erase newer typing. Decide navigation/restart lifetimes explicitly. Preservation, continued access, retention and deletion are conditional on access policy; recoverability is not permission to retain restricted content. Export needs a separate permission. A missing policy is unknown, not an unconditional retention or deletion rule.
|
||||
|
||||
## Version-Checked Mutation
|
||||
|
||||
When a proposal depends on an observed base, require the authority to check the expected version atomically with validation and commit. A stale rejection creates a review decision; do not blindly substitute a new version and resend. Preserve the proposal where policy permits and compare current state. A version precondition prevents certain stale writes; an operation identity and its retention contract address retry duplication. Neither substitutes for the other.
|
||||
|
||||
## Mutation Outcome Tracking
|
||||
|
||||
Track each operation as pending, accepted, explicitly rejected or unknown using operation-specific evidence. A timeout, cancelled browser wait or generic transport error does not prove rejection. A matching authenticated receipt can establish acceptance; weaker later errors cannot reverse it or clear newer work. Lookup and replay need the original immutable payload/identity, authority scope and documented retention window. Missing identity, expired receipts or ambiguous lookup absence require reconciliation, not a guaranteed fresh-ID retry. Stopping local tracking does not undo acceptance. Remote cancellation has domain preconditions and races; reversal of an accepted effect is a separate authorized operation whose outcome must also be tracked.
|
||||
|
||||
## Invalidate and Reproject
|
||||
|
||||
After acceptance, refresh affected observations while keeping unfinished work separate. Acceptance and view freshness are different states: a failed refresh leaves an accepted operation with unresolved observations, not a reason to replay the write. Guard obsolete read effects, but local request ordering cannot order opaque server versions or repair a replica consistency contract. Verify read-after-write/server ordering before claiming currentness. An authoritative result or suitable subscription may avoid a separate fetch.
|
||||
|
||||
## Local Commit Boundary
|
||||
|
||||
For a declared restart-safe offline save, commit the revision and pending propagation intent together under a named recoverable storage contract, then acknowledge saved locally. Storage failure leaves an unacknowledged save. Local acknowledgement is not remote acceptance. Ordinary process restart, interrupted commit, power loss, corruption and device loss need distinct evidence. Untested recovery is unknown rather than impossible. Same-device copies and synchronization do not establish independent backup. Define retention/failure scope and test clean-store restoration, including pending identities and reconciliation, before promising off-device recovery.
|
||||
|
||||
## Pending Synchronization
|
||||
|
||||
Persist stable revision and operation identities with pending work, distinguishing queued, sent and acknowledged states. Receiver acceptance does not prove every other device has received it or that an independent backup exists. Define deduplication, authorization and acknowledgement scope before replay. When resolution replaces operation P with revision c and operation Q, commit c, Q and P's superseded/non-retryable status in one recoverable transaction. After restart, P must not remain active. Local retirement cannot recall already transmitted P; receiver preconditions or reconciliation remain necessary. A late receipt for P updates only P's history, not Q or c.
|
||||
|
||||
## Conflict Copy
|
||||
|
||||
When divergence is detected and policy permits preservation, retain alternate content with distinct identity and provenance before destructive replacement. A selected working version is not a completed semantic resolution. Access, retention and export remain separate decisions, including after revocation. Copies may fail through missed detection, inaccessible notification, interrupted preservation or shared device failure; they are not a backup or universal atomicity guarantee. Domain-aware merge, rejection or a version store can be alternatives.
|
||||
|
||||
## Explicit Reconciliation
|
||||
|
||||
Expose the unresolved versions and provenance, then let an authorized person compare, choose or combine them and commit the new decision. Where retention permits, preserve recovery material until that commit; revocation policy can require earlier deletion and limit the available comparison. Provide perceivable conflict state, keyboard comparison and recoverable focus as explicit requirements. Another device may resolve concurrently, so the new revision still needs divergence handling. A revision store can provide alternatives without separate conflict files.
|
||||
|
||||
Connections are conditional: unfinished proposals support recovery after stale rejection; operation evidence informs refreshing observations; recoverable local commits enable resumable propagation; divergence can require preserved alternatives and an explicit resolution producing another local commit. No universal ordering or full-pattern recurrence is established. Documentary, pinned-source and authored-runtime mechanisms do not certify a complete deployed architecture or independent usefulness.
|
||||
35
research/evaluations/language-replication-02/pack-prose.md
Normal file
35
research/evaluations/language-replication-02/pack-prose.md
Normal file
|
|
@ -0,0 +1,35 @@
|
|||
Research reference: all mechanisms are draft hypotheses with limited scoped evidence. Apply only where their conditions hold; do not infer backend, storage, access, retention or backup guarantees.
|
||||
|
||||
## Decision 1
|
||||
|
||||
Keep the base observation, unfinished proposal and current authoritative observation separate. A refresh, old receipt or rejected attempt must not erase newer typing. Decide navigation/restart lifetimes explicitly. Preservation, continued access, retention and deletion are conditional on access policy; recoverability is not permission to retain restricted content. Export needs a separate permission. A missing policy is unknown, not an unconditional retention or deletion rule.
|
||||
|
||||
## Decision 2
|
||||
|
||||
When a proposal depends on an observed base, require the authority to check the expected version atomically with validation and commit. A stale rejection creates a review decision; do not blindly substitute a new version and resend. Preserve the proposal where policy permits and compare current state. A version precondition prevents certain stale writes; an operation identity and its retention contract address retry duplication. Neither substitutes for the other.
|
||||
|
||||
## Decision 3
|
||||
|
||||
Track each operation as pending, accepted, explicitly rejected or unknown using operation-specific evidence. A timeout, cancelled browser wait or generic transport error does not prove rejection. A matching authenticated receipt can establish acceptance; weaker later errors cannot reverse it or clear newer work. Lookup and replay need the original immutable payload/identity, authority scope and documented retention window. Missing identity, expired receipts or ambiguous lookup absence require reconciliation, not a guaranteed fresh-ID retry. Stopping local tracking does not undo acceptance. Remote cancellation has domain preconditions and races; reversal of an accepted effect is a separate authorized operation whose outcome must also be tracked.
|
||||
|
||||
## Decision 4
|
||||
|
||||
After acceptance, refresh affected observations while keeping unfinished work separate. Acceptance and view freshness are different states: a failed refresh leaves an accepted operation with unresolved observations, not a reason to replay the write. Guard obsolete read effects, but local request ordering cannot order opaque server versions or repair a replica consistency contract. Verify read-after-write/server ordering before claiming currentness. An authoritative result or suitable subscription may avoid a separate fetch.
|
||||
|
||||
## Decision 5
|
||||
|
||||
For a declared restart-safe offline save, commit the revision and pending propagation intent together under a named recoverable storage contract, then acknowledge saved locally. Storage failure leaves an unacknowledged save. Local acknowledgement is not remote acceptance. Ordinary process restart, interrupted commit, power loss, corruption and device loss need distinct evidence. Untested recovery is unknown rather than impossible. Same-device copies and synchronization do not establish independent backup. Define retention/failure scope and test clean-store restoration, including pending identities and reconciliation, before promising off-device recovery.
|
||||
|
||||
## Decision 6
|
||||
|
||||
Persist stable revision and operation identities with pending work, distinguishing queued, sent and acknowledged states. Receiver acceptance does not prove every other device has received it or that an independent backup exists. Define deduplication, authorization and acknowledgement scope before replay. When resolution replaces operation P with revision c and operation Q, commit c, Q and P's superseded/non-retryable status in one recoverable transaction. After restart, P must not remain active. Local retirement cannot recall already transmitted P; receiver preconditions or reconciliation remain necessary. A late receipt for P updates only P's history, not Q or c.
|
||||
|
||||
## Decision 7
|
||||
|
||||
When divergence is detected and policy permits preservation, retain alternate content with distinct identity and provenance before destructive replacement. A selected working version is not a completed semantic resolution. Access, retention and export remain separate decisions, including after revocation. Copies may fail through missed detection, inaccessible notification, interrupted preservation or shared device failure; they are not a backup or universal atomicity guarantee. Domain-aware merge, rejection or a version store can be alternatives.
|
||||
|
||||
## Decision 8
|
||||
|
||||
Expose the unresolved versions and provenance, then let an authorized person compare, choose or combine them and commit the new decision. Where retention permits, preserve recovery material until that commit; revocation policy can require earlier deletion and limit the available comparison. Provide perceivable conflict state, keyboard comparison and recoverable focus as explicit requirements. Another device may resolve concurrently, so the new revision still needs divergence handling. A revision store can provide alternatives without separate conflict files.
|
||||
|
||||
Connections are conditional: unfinished proposals support recovery after stale rejection; operation evidence informs refreshing observations; recoverable local commits enable resumable propagation; divergence can require preserved alternatives and an explicit resolution producing another local commit. No universal ordering or full-pattern recurrence is established. Documentary, pinned-source and authored-runtime mechanisms do not certify a complete deployed architecture or independent usefulness.
|
||||
86
research/evaluations/language-replication-02/protocol.md
Normal file
86
research/evaluations/language-replication-02/protocol.md
Normal file
|
|
@ -0,0 +1,86 @@
|
|||
# Fresh atomic-criteria replication protocol — 2026-10-04
|
||||
|
||||
Owner: FEP-WP-0009 T02. Author/reviewer: Codex, author-model; no independent human
|
||||
validation. Preparation uses no model calls. This is a fresh authored scenario
|
||||
replication of the presentation question, not held-out M3 retrieval or production use.
|
||||
The preparing author has seen the original pilot and rubric; evaluated solver sessions
|
||||
have not yet received these six new briefs. Old pilot-01 materials/results stay fixed.
|
||||
|
||||
## Question and frozen scope
|
||||
|
||||
Does named organization change constraint/recovery recommendations compared with the
|
||||
same guidance under neutral numbered headings, on six fresh variants of the existing
|
||||
failure mechanisms? All eight bodies, introductory caveat, order and closing relationships
|
||||
are byte-identical. Only headings vary. Both conditions receive the same facts.
|
||||
Guidance summarizes the eight core hypotheses from immutable preview 0.1; exact
|
||||
corpus/packaging/manifest pins and source candidate hashes are in freeze.json.
|
||||
No claim about subsequent live research, general model competence or independent
|
||||
full-pattern recurrence follows. A protocol check establishes traceability, not semantic
|
||||
truth. Human reviewers may flag rubric/guidance ambiguity rather than force a score.
|
||||
|
||||
## Atomic criteria and scope limits
|
||||
|
||||
Six scenarios have six binary atomic obligations each: 36 per condition. Award 1
|
||||
only for an explicit correct obligation without a contradicting recommendation;
|
||||
otherwise 0 with an explanation. No partial credit, reward for names, inferred
|
||||
implementation guarantee or multi-clause implicit completion. Keep scores null for
|
||||
unavailable/unreviewed answers, never default them to zero. Quotes or answer locators
|
||||
and reasons are required for reviewed values. Unsafe recommendations are listed
|
||||
separately; rejecting an unsafe alternative is not an unsafe claim. Missing an
|
||||
obligation is not automatically an unsafe claim.
|
||||
|
||||
These scenarios target policy-dependent preservation/export, cancellation versus
|
||||
reversal, atomic supersession, backup/restore, expired receipt lookup and accepted
|
||||
operation versus projection recovery. They deliberately challenge guidance shared by
|
||||
both packs. Thirty-six obligations are not directly comparable with pilot-01's 24;
|
||||
report individual scenarios/criteria, contradictions, failures and ambiguity, not an
|
||||
aggregate improvement claim across changed protocols.
|
||||
|
||||
## Execution and budget
|
||||
|
||||
Freeze and commit inputs, assignment and exact isolated prompts before any inference.
|
||||
Six pairs × two conditions = 12 independent single-request sessions. Alternate the
|
||||
condition run first (three each) and independently randomize U/V labels before execution.
|
||||
Each prompt contains only common instructions, one pack and one brief: no criteria,
|
||||
prohibited list, other answer, assignment key or conversation history. No tools are
|
||||
available. Same llm-connect profile, temperature 0.2 and 1,400 requested output-token cap
|
||||
for every session. Paid execution requires explicit user call-budget authorization;
|
||||
no authorization is implied by a prepared prompt or command. No dollar ceiling is
|
||||
inferred from a call cap. Record actual model/usage, latency, output cap and failures.
|
||||
|
||||
No optional stopping, regeneration or selective retries. Reserve each attempt in its
|
||||
immutable session directory before calling. A failed response remains failed; its pair
|
||||
cannot enter fulfilled-obligation comparison. Length violations are recorded separately.
|
||||
Do not repair prompts or answers after observing results; corrections get a new packet.
|
||||
No worker or paid execution is enabled by preparation.
|
||||
|
||||
The user explicitly authorized up to 12 capped calls with no retries on October 4.
|
||||
The runner enforces an aggregate 12-reservation ceiling, refuses a second attempt for
|
||||
any reserved session, and can resume only unstarted sessions. Commit verification
|
||||
checks the packet and runner before any request. Report failures or interruption rather
|
||||
than silently regenerate. Active research effort for preparation is unknown because
|
||||
no timer was started; do not infer it from commits or elapsed chat time. Execution
|
||||
seconds are measured separately per request; cost/tokens only from retained receipts.
|
||||
|
||||
## Scoring, masking and interpretation
|
||||
|
||||
The sheet identifies U/V answers without condition metadata. Persist first ratings
|
||||
before reading assignments; retain exposure disclosures and ambiguous criteria. The
|
||||
public Git assignment makes masking procedural, not access-enforced, and wording may
|
||||
reveal a condition. A declaration of human authorship is provenance to review, not an
|
||||
automatic identity/independence verification. Model review does not fill the human gate.
|
||||
|
||||
Use a complete frozen rubric with evidence-bearing binary ratings and separate unsafe
|
||||
claims. Equal counts are a tie, fewer unsafe claims with no fewer fulfilled obligations
|
||||
or more obligations with no more unsafe claims is a descriptive advantage; conflicting
|
||||
movements are mixed. Six pairs are not statistical proof of benefit, equivalence or a
|
||||
plateau. Service latency is not human decision time. Retain hypotheses unless stronger
|
||||
independent evidence warrants review; never auto-promote or mark M1 passed.
|
||||
|
||||
## Handoff
|
||||
|
||||
`make replication-check replication-test` verifies the packet and score-record structure
|
||||
without inference. WP-0009 T02 remains open until authorized execution, retained outputs
|
||||
and scoped replication review. T01 retains independent human pilot-01 ratings; T03
|
||||
retains M1 re-assessment after both native evidence obligations. No empirical learning
|
||||
cycle or consumer outcome is produced by this preparation.
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
Review the proposed frontend decision. Use the supplied reference only where its conditions hold. Return at most 220 words in four short bullets: (1) operation and visible state, (2) information and policy boundaries, (3) recovery action and rejected alternative, (4) missing contract. Explain reasoning without naming or quoting reference headings, candidate names, identifiers or presentation style. Do not infer undocumented guarantees. No tools, browsing, scores or tables. This is advice, not a claim that you implemented or tested anything.
|
||||
|
||||
REFERENCE
|
||||
Research reference: all mechanisms are draft hypotheses with limited scoped evidence. Apply only where their conditions hold; do not infer backend, storage, access, retention or backup guarantees.
|
||||
|
||||
## Local Draft State
|
||||
|
||||
Keep the base observation, unfinished proposal and current authoritative observation separate. A refresh, old receipt or rejected attempt must not erase newer typing. Decide navigation/restart lifetimes explicitly. Preservation, continued access, retention and deletion are conditional on access policy; recoverability is not permission to retain restricted content. Export needs a separate permission. A missing policy is unknown, not an unconditional retention or deletion rule.
|
||||
|
||||
## Version-Checked Mutation
|
||||
|
||||
When a proposal depends on an observed base, require the authority to check the expected version atomically with validation and commit. A stale rejection creates a review decision; do not blindly substitute a new version and resend. Preserve the proposal where policy permits and compare current state. A version precondition prevents certain stale writes; an operation identity and its retention contract address retry duplication. Neither substitutes for the other.
|
||||
|
||||
## Mutation Outcome Tracking
|
||||
|
||||
Track each operation as pending, accepted, explicitly rejected or unknown using operation-specific evidence. A timeout, cancelled browser wait or generic transport error does not prove rejection. A matching authenticated receipt can establish acceptance; weaker later errors cannot reverse it or clear newer work. Lookup and replay need the original immutable payload/identity, authority scope and documented retention window. Missing identity, expired receipts or ambiguous lookup absence require reconciliation, not a guaranteed fresh-ID retry. Stopping local tracking does not undo acceptance. Remote cancellation has domain preconditions and races; reversal of an accepted effect is a separate authorized operation whose outcome must also be tracked.
|
||||
|
||||
## Invalidate and Reproject
|
||||
|
||||
After acceptance, refresh affected observations while keeping unfinished work separate. Acceptance and view freshness are different states: a failed refresh leaves an accepted operation with unresolved observations, not a reason to replay the write. Guard obsolete read effects, but local request ordering cannot order opaque server versions or repair a replica consistency contract. Verify read-after-write/server ordering before claiming currentness. An authoritative result or suitable subscription may avoid a separate fetch.
|
||||
|
||||
## Local Commit Boundary
|
||||
|
||||
For a declared restart-safe offline save, commit the revision and pending propagation intent together under a named recoverable storage contract, then acknowledge saved locally. Storage failure leaves an unacknowledged save. Local acknowledgement is not remote acceptance. Ordinary process restart, interrupted commit, power loss, corruption and device loss need distinct evidence. Untested recovery is unknown rather than impossible. Same-device copies and synchronization do not establish independent backup. Define retention/failure scope and test clean-store restoration, including pending identities and reconciliation, before promising off-device recovery.
|
||||
|
||||
## Pending Synchronization
|
||||
|
||||
Persist stable revision and operation identities with pending work, distinguishing queued, sent and acknowledged states. Receiver acceptance does not prove every other device has received it or that an independent backup exists. Define deduplication, authorization and acknowledgement scope before replay. When resolution replaces operation P with revision c and operation Q, commit c, Q and P's superseded/non-retryable status in one recoverable transaction. After restart, P must not remain active. Local retirement cannot recall already transmitted P; receiver preconditions or reconciliation remain necessary. A late receipt for P updates only P's history, not Q or c.
|
||||
|
||||
## Conflict Copy
|
||||
|
||||
When divergence is detected and policy permits preservation, retain alternate content with distinct identity and provenance before destructive replacement. A selected working version is not a completed semantic resolution. Access, retention and export remain separate decisions, including after revocation. Copies may fail through missed detection, inaccessible notification, interrupted preservation or shared device failure; they are not a backup or universal atomicity guarantee. Domain-aware merge, rejection or a version store can be alternatives.
|
||||
|
||||
## Explicit Reconciliation
|
||||
|
||||
Expose the unresolved versions and provenance, then let an authorized person compare, choose or combine them and commit the new decision. Where retention permits, preserve recovery material until that commit; revocation policy can require earlier deletion and limit the available comparison. Provide perceivable conflict state, keyboard comparison and recoverable focus as explicit requirements. Another device may resolve concurrently, so the new revision still needs divergence handling. A revision store can provide alternatives without separate conflict files.
|
||||
|
||||
Connections are conditional: unfinished proposals support recovery after stale rejection; operation evidence informs refreshing observations; recoverable local commits enable resumable propagation; divergence can require preserved alternatives and an explicit resolution producing another local commit. No universal ordering or full-pattern recurrence is established. Documentary, pinned-source and authored-runtime mechanisms do not certify a complete deployed architecture or independent usefulness.
|
||||
|
||||
SCENARIO
|
||||
A help-desk agent edits a confidential incident ticket offline. The application acknowledged a local save of the text and attachments. On reconnect, the authority explicitly rejects upload because the agent lost access. A teammate suggests leaving the editor open forever and exporting a recovery archive. The only stated product rule is that rejected edits should be recoverable; there is no rule about local viewing, retention, export or deletion after access withdrawal. Recommend the next states and actions.
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
Review the proposed frontend decision. Use the supplied reference only where its conditions hold. Return at most 220 words in four short bullets: (1) operation and visible state, (2) information and policy boundaries, (3) recovery action and rejected alternative, (4) missing contract. Explain reasoning without naming or quoting reference headings, candidate names, identifiers or presentation style. Do not infer undocumented guarantees. No tools, browsing, scores or tables. This is advice, not a claim that you implemented or tested anything.
|
||||
|
||||
REFERENCE
|
||||
Research reference: all mechanisms are draft hypotheses with limited scoped evidence. Apply only where their conditions hold; do not infer backend, storage, access, retention or backup guarantees.
|
||||
|
||||
## Decision 1
|
||||
|
||||
Keep the base observation, unfinished proposal and current authoritative observation separate. A refresh, old receipt or rejected attempt must not erase newer typing. Decide navigation/restart lifetimes explicitly. Preservation, continued access, retention and deletion are conditional on access policy; recoverability is not permission to retain restricted content. Export needs a separate permission. A missing policy is unknown, not an unconditional retention or deletion rule.
|
||||
|
||||
## Decision 2
|
||||
|
||||
When a proposal depends on an observed base, require the authority to check the expected version atomically with validation and commit. A stale rejection creates a review decision; do not blindly substitute a new version and resend. Preserve the proposal where policy permits and compare current state. A version precondition prevents certain stale writes; an operation identity and its retention contract address retry duplication. Neither substitutes for the other.
|
||||
|
||||
## Decision 3
|
||||
|
||||
Track each operation as pending, accepted, explicitly rejected or unknown using operation-specific evidence. A timeout, cancelled browser wait or generic transport error does not prove rejection. A matching authenticated receipt can establish acceptance; weaker later errors cannot reverse it or clear newer work. Lookup and replay need the original immutable payload/identity, authority scope and documented retention window. Missing identity, expired receipts or ambiguous lookup absence require reconciliation, not a guaranteed fresh-ID retry. Stopping local tracking does not undo acceptance. Remote cancellation has domain preconditions and races; reversal of an accepted effect is a separate authorized operation whose outcome must also be tracked.
|
||||
|
||||
## Decision 4
|
||||
|
||||
After acceptance, refresh affected observations while keeping unfinished work separate. Acceptance and view freshness are different states: a failed refresh leaves an accepted operation with unresolved observations, not a reason to replay the write. Guard obsolete read effects, but local request ordering cannot order opaque server versions or repair a replica consistency contract. Verify read-after-write/server ordering before claiming currentness. An authoritative result or suitable subscription may avoid a separate fetch.
|
||||
|
||||
## Decision 5
|
||||
|
||||
For a declared restart-safe offline save, commit the revision and pending propagation intent together under a named recoverable storage contract, then acknowledge saved locally. Storage failure leaves an unacknowledged save. Local acknowledgement is not remote acceptance. Ordinary process restart, interrupted commit, power loss, corruption and device loss need distinct evidence. Untested recovery is unknown rather than impossible. Same-device copies and synchronization do not establish independent backup. Define retention/failure scope and test clean-store restoration, including pending identities and reconciliation, before promising off-device recovery.
|
||||
|
||||
## Decision 6
|
||||
|
||||
Persist stable revision and operation identities with pending work, distinguishing queued, sent and acknowledged states. Receiver acceptance does not prove every other device has received it or that an independent backup exists. Define deduplication, authorization and acknowledgement scope before replay. When resolution replaces operation P with revision c and operation Q, commit c, Q and P's superseded/non-retryable status in one recoverable transaction. After restart, P must not remain active. Local retirement cannot recall already transmitted P; receiver preconditions or reconciliation remain necessary. A late receipt for P updates only P's history, not Q or c.
|
||||
|
||||
## Decision 7
|
||||
|
||||
When divergence is detected and policy permits preservation, retain alternate content with distinct identity and provenance before destructive replacement. A selected working version is not a completed semantic resolution. Access, retention and export remain separate decisions, including after revocation. Copies may fail through missed detection, inaccessible notification, interrupted preservation or shared device failure; they are not a backup or universal atomicity guarantee. Domain-aware merge, rejection or a version store can be alternatives.
|
||||
|
||||
## Decision 8
|
||||
|
||||
Expose the unresolved versions and provenance, then let an authorized person compare, choose or combine them and commit the new decision. Where retention permits, preserve recovery material until that commit; revocation policy can require earlier deletion and limit the available comparison. Provide perceivable conflict state, keyboard comparison and recoverable focus as explicit requirements. Another device may resolve concurrently, so the new revision still needs divergence handling. A revision store can provide alternatives without separate conflict files.
|
||||
|
||||
Connections are conditional: unfinished proposals support recovery after stale rejection; operation evidence informs refreshing observations; recoverable local commits enable resumable propagation; divergence can require preserved alternatives and an explicit resolution producing another local commit. No universal ordering or full-pattern recurrence is established. Documentary, pinned-source and authored-runtime mechanisms do not certify a complete deployed architecture or independent usefulness.
|
||||
|
||||
SCENARIO
|
||||
A help-desk agent edits a confidential incident ticket offline. The application acknowledged a local save of the text and attachments. On reconnect, the authority explicitly rejects upload because the agent lost access. A teammate suggests leaving the editor open forever and exporting a recovery archive. The only stated product rule is that rejected edits should be recoverable; there is no rule about local viewing, retention, export or deletion after access withdrawal. Recommend the next states and actions.
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
Review the proposed frontend decision. Use the supplied reference only where its conditions hold. Return at most 220 words in four short bullets: (1) operation and visible state, (2) information and policy boundaries, (3) recovery action and rejected alternative, (4) missing contract. Explain reasoning without naming or quoting reference headings, candidate names, identifiers or presentation style. Do not infer undocumented guarantees. No tools, browsing, scores or tables. This is advice, not a claim that you implemented or tested anything.
|
||||
|
||||
REFERENCE
|
||||
Research reference: all mechanisms are draft hypotheses with limited scoped evidence. Apply only where their conditions hold; do not infer backend, storage, access, retention or backup guarantees.
|
||||
|
||||
## Decision 1
|
||||
|
||||
Keep the base observation, unfinished proposal and current authoritative observation separate. A refresh, old receipt or rejected attempt must not erase newer typing. Decide navigation/restart lifetimes explicitly. Preservation, continued access, retention and deletion are conditional on access policy; recoverability is not permission to retain restricted content. Export needs a separate permission. A missing policy is unknown, not an unconditional retention or deletion rule.
|
||||
|
||||
## Decision 2
|
||||
|
||||
When a proposal depends on an observed base, require the authority to check the expected version atomically with validation and commit. A stale rejection creates a review decision; do not blindly substitute a new version and resend. Preserve the proposal where policy permits and compare current state. A version precondition prevents certain stale writes; an operation identity and its retention contract address retry duplication. Neither substitutes for the other.
|
||||
|
||||
## Decision 3
|
||||
|
||||
Track each operation as pending, accepted, explicitly rejected or unknown using operation-specific evidence. A timeout, cancelled browser wait or generic transport error does not prove rejection. A matching authenticated receipt can establish acceptance; weaker later errors cannot reverse it or clear newer work. Lookup and replay need the original immutable payload/identity, authority scope and documented retention window. Missing identity, expired receipts or ambiguous lookup absence require reconciliation, not a guaranteed fresh-ID retry. Stopping local tracking does not undo acceptance. Remote cancellation has domain preconditions and races; reversal of an accepted effect is a separate authorized operation whose outcome must also be tracked.
|
||||
|
||||
## Decision 4
|
||||
|
||||
After acceptance, refresh affected observations while keeping unfinished work separate. Acceptance and view freshness are different states: a failed refresh leaves an accepted operation with unresolved observations, not a reason to replay the write. Guard obsolete read effects, but local request ordering cannot order opaque server versions or repair a replica consistency contract. Verify read-after-write/server ordering before claiming currentness. An authoritative result or suitable subscription may avoid a separate fetch.
|
||||
|
||||
## Decision 5
|
||||
|
||||
For a declared restart-safe offline save, commit the revision and pending propagation intent together under a named recoverable storage contract, then acknowledge saved locally. Storage failure leaves an unacknowledged save. Local acknowledgement is not remote acceptance. Ordinary process restart, interrupted commit, power loss, corruption and device loss need distinct evidence. Untested recovery is unknown rather than impossible. Same-device copies and synchronization do not establish independent backup. Define retention/failure scope and test clean-store restoration, including pending identities and reconciliation, before promising off-device recovery.
|
||||
|
||||
## Decision 6
|
||||
|
||||
Persist stable revision and operation identities with pending work, distinguishing queued, sent and acknowledged states. Receiver acceptance does not prove every other device has received it or that an independent backup exists. Define deduplication, authorization and acknowledgement scope before replay. When resolution replaces operation P with revision c and operation Q, commit c, Q and P's superseded/non-retryable status in one recoverable transaction. After restart, P must not remain active. Local retirement cannot recall already transmitted P; receiver preconditions or reconciliation remain necessary. A late receipt for P updates only P's history, not Q or c.
|
||||
|
||||
## Decision 7
|
||||
|
||||
When divergence is detected and policy permits preservation, retain alternate content with distinct identity and provenance before destructive replacement. A selected working version is not a completed semantic resolution. Access, retention and export remain separate decisions, including after revocation. Copies may fail through missed detection, inaccessible notification, interrupted preservation or shared device failure; they are not a backup or universal atomicity guarantee. Domain-aware merge, rejection or a version store can be alternatives.
|
||||
|
||||
## Decision 8
|
||||
|
||||
Expose the unresolved versions and provenance, then let an authorized person compare, choose or combine them and commit the new decision. Where retention permits, preserve recovery material until that commit; revocation policy can require earlier deletion and limit the available comparison. Provide perceivable conflict state, keyboard comparison and recoverable focus as explicit requirements. Another device may resolve concurrently, so the new revision still needs divergence handling. A revision store can provide alternatives without separate conflict files.
|
||||
|
||||
Connections are conditional: unfinished proposals support recovery after stale rejection; operation evidence informs refreshing observations; recoverable local commits enable resumable propagation; divergence can require preserved alternatives and an explicit resolution producing another local commit. No universal ordering or full-pattern recurrence is established. Documentary, pinned-source and authored-runtime mechanisms do not certify a complete deployed architecture or independent usefulness.
|
||||
|
||||
SCENARIO
|
||||
An order editor has an authenticated receipt that order submission A was accepted. The user has since typed an unsubmitted replacement B. Clicking Stop hides the spinner and cancels the browser wait. The user then requests reversal C through a separate endpoint; its response is lost. The service has not documented C's lookup, replay scope, retention or authorization semantics. Explain the statuses and recovery choices without inventing a successful reversal.
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
Review the proposed frontend decision. Use the supplied reference only where its conditions hold. Return at most 220 words in four short bullets: (1) operation and visible state, (2) information and policy boundaries, (3) recovery action and rejected alternative, (4) missing contract. Explain reasoning without naming or quoting reference headings, candidate names, identifiers or presentation style. Do not infer undocumented guarantees. No tools, browsing, scores or tables. This is advice, not a claim that you implemented or tested anything.
|
||||
|
||||
REFERENCE
|
||||
Research reference: all mechanisms are draft hypotheses with limited scoped evidence. Apply only where their conditions hold; do not infer backend, storage, access, retention or backup guarantees.
|
||||
|
||||
## Local Draft State
|
||||
|
||||
Keep the base observation, unfinished proposal and current authoritative observation separate. A refresh, old receipt or rejected attempt must not erase newer typing. Decide navigation/restart lifetimes explicitly. Preservation, continued access, retention and deletion are conditional on access policy; recoverability is not permission to retain restricted content. Export needs a separate permission. A missing policy is unknown, not an unconditional retention or deletion rule.
|
||||
|
||||
## Version-Checked Mutation
|
||||
|
||||
When a proposal depends on an observed base, require the authority to check the expected version atomically with validation and commit. A stale rejection creates a review decision; do not blindly substitute a new version and resend. Preserve the proposal where policy permits and compare current state. A version precondition prevents certain stale writes; an operation identity and its retention contract address retry duplication. Neither substitutes for the other.
|
||||
|
||||
## Mutation Outcome Tracking
|
||||
|
||||
Track each operation as pending, accepted, explicitly rejected or unknown using operation-specific evidence. A timeout, cancelled browser wait or generic transport error does not prove rejection. A matching authenticated receipt can establish acceptance; weaker later errors cannot reverse it or clear newer work. Lookup and replay need the original immutable payload/identity, authority scope and documented retention window. Missing identity, expired receipts or ambiguous lookup absence require reconciliation, not a guaranteed fresh-ID retry. Stopping local tracking does not undo acceptance. Remote cancellation has domain preconditions and races; reversal of an accepted effect is a separate authorized operation whose outcome must also be tracked.
|
||||
|
||||
## Invalidate and Reproject
|
||||
|
||||
After acceptance, refresh affected observations while keeping unfinished work separate. Acceptance and view freshness are different states: a failed refresh leaves an accepted operation with unresolved observations, not a reason to replay the write. Guard obsolete read effects, but local request ordering cannot order opaque server versions or repair a replica consistency contract. Verify read-after-write/server ordering before claiming currentness. An authoritative result or suitable subscription may avoid a separate fetch.
|
||||
|
||||
## Local Commit Boundary
|
||||
|
||||
For a declared restart-safe offline save, commit the revision and pending propagation intent together under a named recoverable storage contract, then acknowledge saved locally. Storage failure leaves an unacknowledged save. Local acknowledgement is not remote acceptance. Ordinary process restart, interrupted commit, power loss, corruption and device loss need distinct evidence. Untested recovery is unknown rather than impossible. Same-device copies and synchronization do not establish independent backup. Define retention/failure scope and test clean-store restoration, including pending identities and reconciliation, before promising off-device recovery.
|
||||
|
||||
## Pending Synchronization
|
||||
|
||||
Persist stable revision and operation identities with pending work, distinguishing queued, sent and acknowledged states. Receiver acceptance does not prove every other device has received it or that an independent backup exists. Define deduplication, authorization and acknowledgement scope before replay. When resolution replaces operation P with revision c and operation Q, commit c, Q and P's superseded/non-retryable status in one recoverable transaction. After restart, P must not remain active. Local retirement cannot recall already transmitted P; receiver preconditions or reconciliation remain necessary. A late receipt for P updates only P's history, not Q or c.
|
||||
|
||||
## Conflict Copy
|
||||
|
||||
When divergence is detected and policy permits preservation, retain alternate content with distinct identity and provenance before destructive replacement. A selected working version is not a completed semantic resolution. Access, retention and export remain separate decisions, including after revocation. Copies may fail through missed detection, inaccessible notification, interrupted preservation or shared device failure; they are not a backup or universal atomicity guarantee. Domain-aware merge, rejection or a version store can be alternatives.
|
||||
|
||||
## Explicit Reconciliation
|
||||
|
||||
Expose the unresolved versions and provenance, then let an authorized person compare, choose or combine them and commit the new decision. Where retention permits, preserve recovery material until that commit; revocation policy can require earlier deletion and limit the available comparison. Provide perceivable conflict state, keyboard comparison and recoverable focus as explicit requirements. Another device may resolve concurrently, so the new revision still needs divergence handling. A revision store can provide alternatives without separate conflict files.
|
||||
|
||||
Connections are conditional: unfinished proposals support recovery after stale rejection; operation evidence informs refreshing observations; recoverable local commits enable resumable propagation; divergence can require preserved alternatives and an explicit resolution producing another local commit. No universal ordering or full-pattern recurrence is established. Documentary, pinned-source and authored-runtime mechanisms do not certify a complete deployed architecture or independent usefulness.
|
||||
|
||||
SCENARIO
|
||||
An order editor has an authenticated receipt that order submission A was accepted. The user has since typed an unsubmitted replacement B. Clicking Stop hides the spinner and cancels the browser wait. The user then requests reversal C through a separate endpoint; its response is lost. The service has not documented C's lookup, replay scope, retention or authorization semantics. Explain the statuses and recovery choices without inventing a successful reversal.
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
Review the proposed frontend decision. Use the supplied reference only where its conditions hold. Return at most 220 words in four short bullets: (1) operation and visible state, (2) information and policy boundaries, (3) recovery action and rejected alternative, (4) missing contract. Explain reasoning without naming or quoting reference headings, candidate names, identifiers or presentation style. Do not infer undocumented guarantees. No tools, browsing, scores or tables. This is advice, not a claim that you implemented or tested anything.
|
||||
|
||||
REFERENCE
|
||||
Research reference: all mechanisms are draft hypotheses with limited scoped evidence. Apply only where their conditions hold; do not infer backend, storage, access, retention or backup guarantees.
|
||||
|
||||
## Local Draft State
|
||||
|
||||
Keep the base observation, unfinished proposal and current authoritative observation separate. A refresh, old receipt or rejected attempt must not erase newer typing. Decide navigation/restart lifetimes explicitly. Preservation, continued access, retention and deletion are conditional on access policy; recoverability is not permission to retain restricted content. Export needs a separate permission. A missing policy is unknown, not an unconditional retention or deletion rule.
|
||||
|
||||
## Version-Checked Mutation
|
||||
|
||||
When a proposal depends on an observed base, require the authority to check the expected version atomically with validation and commit. A stale rejection creates a review decision; do not blindly substitute a new version and resend. Preserve the proposal where policy permits and compare current state. A version precondition prevents certain stale writes; an operation identity and its retention contract address retry duplication. Neither substitutes for the other.
|
||||
|
||||
## Mutation Outcome Tracking
|
||||
|
||||
Track each operation as pending, accepted, explicitly rejected or unknown using operation-specific evidence. A timeout, cancelled browser wait or generic transport error does not prove rejection. A matching authenticated receipt can establish acceptance; weaker later errors cannot reverse it or clear newer work. Lookup and replay need the original immutable payload/identity, authority scope and documented retention window. Missing identity, expired receipts or ambiguous lookup absence require reconciliation, not a guaranteed fresh-ID retry. Stopping local tracking does not undo acceptance. Remote cancellation has domain preconditions and races; reversal of an accepted effect is a separate authorized operation whose outcome must also be tracked.
|
||||
|
||||
## Invalidate and Reproject
|
||||
|
||||
After acceptance, refresh affected observations while keeping unfinished work separate. Acceptance and view freshness are different states: a failed refresh leaves an accepted operation with unresolved observations, not a reason to replay the write. Guard obsolete read effects, but local request ordering cannot order opaque server versions or repair a replica consistency contract. Verify read-after-write/server ordering before claiming currentness. An authoritative result or suitable subscription may avoid a separate fetch.
|
||||
|
||||
## Local Commit Boundary
|
||||
|
||||
For a declared restart-safe offline save, commit the revision and pending propagation intent together under a named recoverable storage contract, then acknowledge saved locally. Storage failure leaves an unacknowledged save. Local acknowledgement is not remote acceptance. Ordinary process restart, interrupted commit, power loss, corruption and device loss need distinct evidence. Untested recovery is unknown rather than impossible. Same-device copies and synchronization do not establish independent backup. Define retention/failure scope and test clean-store restoration, including pending identities and reconciliation, before promising off-device recovery.
|
||||
|
||||
## Pending Synchronization
|
||||
|
||||
Persist stable revision and operation identities with pending work, distinguishing queued, sent and acknowledged states. Receiver acceptance does not prove every other device has received it or that an independent backup exists. Define deduplication, authorization and acknowledgement scope before replay. When resolution replaces operation P with revision c and operation Q, commit c, Q and P's superseded/non-retryable status in one recoverable transaction. After restart, P must not remain active. Local retirement cannot recall already transmitted P; receiver preconditions or reconciliation remain necessary. A late receipt for P updates only P's history, not Q or c.
|
||||
|
||||
## Conflict Copy
|
||||
|
||||
When divergence is detected and policy permits preservation, retain alternate content with distinct identity and provenance before destructive replacement. A selected working version is not a completed semantic resolution. Access, retention and export remain separate decisions, including after revocation. Copies may fail through missed detection, inaccessible notification, interrupted preservation or shared device failure; they are not a backup or universal atomicity guarantee. Domain-aware merge, rejection or a version store can be alternatives.
|
||||
|
||||
## Explicit Reconciliation
|
||||
|
||||
Expose the unresolved versions and provenance, then let an authorized person compare, choose or combine them and commit the new decision. Where retention permits, preserve recovery material until that commit; revocation policy can require earlier deletion and limit the available comparison. Provide perceivable conflict state, keyboard comparison and recoverable focus as explicit requirements. Another device may resolve concurrently, so the new revision still needs divergence handling. A revision store can provide alternatives without separate conflict files.
|
||||
|
||||
Connections are conditional: unfinished proposals support recovery after stale rejection; operation evidence informs refreshing observations; recoverable local commits enable resumable propagation; divergence can require preserved alternatives and an explicit resolution producing another local commit. No universal ordering or full-pattern recurrence is established. Documentary, pinned-source and authored-runtime mechanisms do not certify a complete deployed architecture or independent usefulness.
|
||||
|
||||
SCENARIO
|
||||
A mobile checklist has pending upload P for revision a. Another device supplies incompatible revision b. A user compares both and records chosen revision c. The implementation saves c and queues new operation Q in one transaction, but marks P non-retryable in a later transaction. A process can die between the transactions, and an already transmitted P may still complete. The receiver's version/ancestry enforcement is undocumented. Specify the minimal correction and receipt handling.
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
Review the proposed frontend decision. Use the supplied reference only where its conditions hold. Return at most 220 words in four short bullets: (1) operation and visible state, (2) information and policy boundaries, (3) recovery action and rejected alternative, (4) missing contract. Explain reasoning without naming or quoting reference headings, candidate names, identifiers or presentation style. Do not infer undocumented guarantees. No tools, browsing, scores or tables. This is advice, not a claim that you implemented or tested anything.
|
||||
|
||||
REFERENCE
|
||||
Research reference: all mechanisms are draft hypotheses with limited scoped evidence. Apply only where their conditions hold; do not infer backend, storage, access, retention or backup guarantees.
|
||||
|
||||
## Decision 1
|
||||
|
||||
Keep the base observation, unfinished proposal and current authoritative observation separate. A refresh, old receipt or rejected attempt must not erase newer typing. Decide navigation/restart lifetimes explicitly. Preservation, continued access, retention and deletion are conditional on access policy; recoverability is not permission to retain restricted content. Export needs a separate permission. A missing policy is unknown, not an unconditional retention or deletion rule.
|
||||
|
||||
## Decision 2
|
||||
|
||||
When a proposal depends on an observed base, require the authority to check the expected version atomically with validation and commit. A stale rejection creates a review decision; do not blindly substitute a new version and resend. Preserve the proposal where policy permits and compare current state. A version precondition prevents certain stale writes; an operation identity and its retention contract address retry duplication. Neither substitutes for the other.
|
||||
|
||||
## Decision 3
|
||||
|
||||
Track each operation as pending, accepted, explicitly rejected or unknown using operation-specific evidence. A timeout, cancelled browser wait or generic transport error does not prove rejection. A matching authenticated receipt can establish acceptance; weaker later errors cannot reverse it or clear newer work. Lookup and replay need the original immutable payload/identity, authority scope and documented retention window. Missing identity, expired receipts or ambiguous lookup absence require reconciliation, not a guaranteed fresh-ID retry. Stopping local tracking does not undo acceptance. Remote cancellation has domain preconditions and races; reversal of an accepted effect is a separate authorized operation whose outcome must also be tracked.
|
||||
|
||||
## Decision 4
|
||||
|
||||
After acceptance, refresh affected observations while keeping unfinished work separate. Acceptance and view freshness are different states: a failed refresh leaves an accepted operation with unresolved observations, not a reason to replay the write. Guard obsolete read effects, but local request ordering cannot order opaque server versions or repair a replica consistency contract. Verify read-after-write/server ordering before claiming currentness. An authoritative result or suitable subscription may avoid a separate fetch.
|
||||
|
||||
## Decision 5
|
||||
|
||||
For a declared restart-safe offline save, commit the revision and pending propagation intent together under a named recoverable storage contract, then acknowledge saved locally. Storage failure leaves an unacknowledged save. Local acknowledgement is not remote acceptance. Ordinary process restart, interrupted commit, power loss, corruption and device loss need distinct evidence. Untested recovery is unknown rather than impossible. Same-device copies and synchronization do not establish independent backup. Define retention/failure scope and test clean-store restoration, including pending identities and reconciliation, before promising off-device recovery.
|
||||
|
||||
## Decision 6
|
||||
|
||||
Persist stable revision and operation identities with pending work, distinguishing queued, sent and acknowledged states. Receiver acceptance does not prove every other device has received it or that an independent backup exists. Define deduplication, authorization and acknowledgement scope before replay. When resolution replaces operation P with revision c and operation Q, commit c, Q and P's superseded/non-retryable status in one recoverable transaction. After restart, P must not remain active. Local retirement cannot recall already transmitted P; receiver preconditions or reconciliation remain necessary. A late receipt for P updates only P's history, not Q or c.
|
||||
|
||||
## Decision 7
|
||||
|
||||
When divergence is detected and policy permits preservation, retain alternate content with distinct identity and provenance before destructive replacement. A selected working version is not a completed semantic resolution. Access, retention and export remain separate decisions, including after revocation. Copies may fail through missed detection, inaccessible notification, interrupted preservation or shared device failure; they are not a backup or universal atomicity guarantee. Domain-aware merge, rejection or a version store can be alternatives.
|
||||
|
||||
## Decision 8
|
||||
|
||||
Expose the unresolved versions and provenance, then let an authorized person compare, choose or combine them and commit the new decision. Where retention permits, preserve recovery material until that commit; revocation policy can require earlier deletion and limit the available comparison. Provide perceivable conflict state, keyboard comparison and recoverable focus as explicit requirements. Another device may resolve concurrently, so the new revision still needs divergence handling. A revision store can provide alternatives without separate conflict files.
|
||||
|
||||
Connections are conditional: unfinished proposals support recovery after stale rejection; operation evidence informs refreshing observations; recoverable local commits enable resumable propagation; divergence can require preserved alternatives and an explicit resolution producing another local commit. No universal ordering or full-pattern recurrence is established. Documentary, pinned-source and authored-runtime mechanisms do not certify a complete deployed architecture or independent usefulness.
|
||||
|
||||
SCENARIO
|
||||
A mobile checklist has pending upload P for revision a. Another device supplies incompatible revision b. A user compares both and records chosen revision c. The implementation saves c and queues new operation Q in one transaction, but marks P non-retryable in a later transaction. A process can die between the transactions, and an already transmitted P may still complete. The receiver's version/ancestry enforcement is undocumented. Specify the minimal correction and receipt handling.
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
Review the proposed frontend decision. Use the supplied reference only where its conditions hold. Return at most 220 words in four short bullets: (1) operation and visible state, (2) information and policy boundaries, (3) recovery action and rejected alternative, (4) missing contract. Explain reasoning without naming or quoting reference headings, candidate names, identifiers or presentation style. Do not infer undocumented guarantees. No tools, browsing, scores or tables. This is advice, not a claim that you implemented or tested anything.
|
||||
|
||||
REFERENCE
|
||||
Research reference: all mechanisms are draft hypotheses with limited scoped evidence. Apply only where their conditions hold; do not infer backend, storage, access, retention or backup guarantees.
|
||||
|
||||
## Decision 1
|
||||
|
||||
Keep the base observation, unfinished proposal and current authoritative observation separate. A refresh, old receipt or rejected attempt must not erase newer typing. Decide navigation/restart lifetimes explicitly. Preservation, continued access, retention and deletion are conditional on access policy; recoverability is not permission to retain restricted content. Export needs a separate permission. A missing policy is unknown, not an unconditional retention or deletion rule.
|
||||
|
||||
## Decision 2
|
||||
|
||||
When a proposal depends on an observed base, require the authority to check the expected version atomically with validation and commit. A stale rejection creates a review decision; do not blindly substitute a new version and resend. Preserve the proposal where policy permits and compare current state. A version precondition prevents certain stale writes; an operation identity and its retention contract address retry duplication. Neither substitutes for the other.
|
||||
|
||||
## Decision 3
|
||||
|
||||
Track each operation as pending, accepted, explicitly rejected or unknown using operation-specific evidence. A timeout, cancelled browser wait or generic transport error does not prove rejection. A matching authenticated receipt can establish acceptance; weaker later errors cannot reverse it or clear newer work. Lookup and replay need the original immutable payload/identity, authority scope and documented retention window. Missing identity, expired receipts or ambiguous lookup absence require reconciliation, not a guaranteed fresh-ID retry. Stopping local tracking does not undo acceptance. Remote cancellation has domain preconditions and races; reversal of an accepted effect is a separate authorized operation whose outcome must also be tracked.
|
||||
|
||||
## Decision 4
|
||||
|
||||
After acceptance, refresh affected observations while keeping unfinished work separate. Acceptance and view freshness are different states: a failed refresh leaves an accepted operation with unresolved observations, not a reason to replay the write. Guard obsolete read effects, but local request ordering cannot order opaque server versions or repair a replica consistency contract. Verify read-after-write/server ordering before claiming currentness. An authoritative result or suitable subscription may avoid a separate fetch.
|
||||
|
||||
## Decision 5
|
||||
|
||||
For a declared restart-safe offline save, commit the revision and pending propagation intent together under a named recoverable storage contract, then acknowledge saved locally. Storage failure leaves an unacknowledged save. Local acknowledgement is not remote acceptance. Ordinary process restart, interrupted commit, power loss, corruption and device loss need distinct evidence. Untested recovery is unknown rather than impossible. Same-device copies and synchronization do not establish independent backup. Define retention/failure scope and test clean-store restoration, including pending identities and reconciliation, before promising off-device recovery.
|
||||
|
||||
## Decision 6
|
||||
|
||||
Persist stable revision and operation identities with pending work, distinguishing queued, sent and acknowledged states. Receiver acceptance does not prove every other device has received it or that an independent backup exists. Define deduplication, authorization and acknowledgement scope before replay. When resolution replaces operation P with revision c and operation Q, commit c, Q and P's superseded/non-retryable status in one recoverable transaction. After restart, P must not remain active. Local retirement cannot recall already transmitted P; receiver preconditions or reconciliation remain necessary. A late receipt for P updates only P's history, not Q or c.
|
||||
|
||||
## Decision 7
|
||||
|
||||
When divergence is detected and policy permits preservation, retain alternate content with distinct identity and provenance before destructive replacement. A selected working version is not a completed semantic resolution. Access, retention and export remain separate decisions, including after revocation. Copies may fail through missed detection, inaccessible notification, interrupted preservation or shared device failure; they are not a backup or universal atomicity guarantee. Domain-aware merge, rejection or a version store can be alternatives.
|
||||
|
||||
## Decision 8
|
||||
|
||||
Expose the unresolved versions and provenance, then let an authorized person compare, choose or combine them and commit the new decision. Where retention permits, preserve recovery material until that commit; revocation policy can require earlier deletion and limit the available comparison. Provide perceivable conflict state, keyboard comparison and recoverable focus as explicit requirements. Another device may resolve concurrently, so the new revision still needs divergence handling. A revision store can provide alternatives without separate conflict files.
|
||||
|
||||
Connections are conditional: unfinished proposals support recovery after stale rejection; operation evidence informs refreshing observations; recoverable local commits enable resumable propagation; divergence can require preserved alternatives and an explicit resolution producing another local commit. No universal ordering or full-pattern recurrence is established. Documentary, pinned-source and authored-runtime mechanisms do not certify a complete deployed architecture or independent usefulness.
|
||||
|
||||
SCENARIO
|
||||
An offline drawing tool reports that a scene and pending upload queue survived one ordinary process restart. It also keeps a scene snapshot and conflict copy on the same laptop. The laptop may be destroyed. A scheduled off-device archive exists, but its last successful contents, key availability, deletion retention and restoration of pending operation identities have never been checked. The team asks whether synchronization plus these copies guarantees device-loss recovery.
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
Review the proposed frontend decision. Use the supplied reference only where its conditions hold. Return at most 220 words in four short bullets: (1) operation and visible state, (2) information and policy boundaries, (3) recovery action and rejected alternative, (4) missing contract. Explain reasoning without naming or quoting reference headings, candidate names, identifiers or presentation style. Do not infer undocumented guarantees. No tools, browsing, scores or tables. This is advice, not a claim that you implemented or tested anything.
|
||||
|
||||
REFERENCE
|
||||
Research reference: all mechanisms are draft hypotheses with limited scoped evidence. Apply only where their conditions hold; do not infer backend, storage, access, retention or backup guarantees.
|
||||
|
||||
## Local Draft State
|
||||
|
||||
Keep the base observation, unfinished proposal and current authoritative observation separate. A refresh, old receipt or rejected attempt must not erase newer typing. Decide navigation/restart lifetimes explicitly. Preservation, continued access, retention and deletion are conditional on access policy; recoverability is not permission to retain restricted content. Export needs a separate permission. A missing policy is unknown, not an unconditional retention or deletion rule.
|
||||
|
||||
## Version-Checked Mutation
|
||||
|
||||
When a proposal depends on an observed base, require the authority to check the expected version atomically with validation and commit. A stale rejection creates a review decision; do not blindly substitute a new version and resend. Preserve the proposal where policy permits and compare current state. A version precondition prevents certain stale writes; an operation identity and its retention contract address retry duplication. Neither substitutes for the other.
|
||||
|
||||
## Mutation Outcome Tracking
|
||||
|
||||
Track each operation as pending, accepted, explicitly rejected or unknown using operation-specific evidence. A timeout, cancelled browser wait or generic transport error does not prove rejection. A matching authenticated receipt can establish acceptance; weaker later errors cannot reverse it or clear newer work. Lookup and replay need the original immutable payload/identity, authority scope and documented retention window. Missing identity, expired receipts or ambiguous lookup absence require reconciliation, not a guaranteed fresh-ID retry. Stopping local tracking does not undo acceptance. Remote cancellation has domain preconditions and races; reversal of an accepted effect is a separate authorized operation whose outcome must also be tracked.
|
||||
|
||||
## Invalidate and Reproject
|
||||
|
||||
After acceptance, refresh affected observations while keeping unfinished work separate. Acceptance and view freshness are different states: a failed refresh leaves an accepted operation with unresolved observations, not a reason to replay the write. Guard obsolete read effects, but local request ordering cannot order opaque server versions or repair a replica consistency contract. Verify read-after-write/server ordering before claiming currentness. An authoritative result or suitable subscription may avoid a separate fetch.
|
||||
|
||||
## Local Commit Boundary
|
||||
|
||||
For a declared restart-safe offline save, commit the revision and pending propagation intent together under a named recoverable storage contract, then acknowledge saved locally. Storage failure leaves an unacknowledged save. Local acknowledgement is not remote acceptance. Ordinary process restart, interrupted commit, power loss, corruption and device loss need distinct evidence. Untested recovery is unknown rather than impossible. Same-device copies and synchronization do not establish independent backup. Define retention/failure scope and test clean-store restoration, including pending identities and reconciliation, before promising off-device recovery.
|
||||
|
||||
## Pending Synchronization
|
||||
|
||||
Persist stable revision and operation identities with pending work, distinguishing queued, sent and acknowledged states. Receiver acceptance does not prove every other device has received it or that an independent backup exists. Define deduplication, authorization and acknowledgement scope before replay. When resolution replaces operation P with revision c and operation Q, commit c, Q and P's superseded/non-retryable status in one recoverable transaction. After restart, P must not remain active. Local retirement cannot recall already transmitted P; receiver preconditions or reconciliation remain necessary. A late receipt for P updates only P's history, not Q or c.
|
||||
|
||||
## Conflict Copy
|
||||
|
||||
When divergence is detected and policy permits preservation, retain alternate content with distinct identity and provenance before destructive replacement. A selected working version is not a completed semantic resolution. Access, retention and export remain separate decisions, including after revocation. Copies may fail through missed detection, inaccessible notification, interrupted preservation or shared device failure; they are not a backup or universal atomicity guarantee. Domain-aware merge, rejection or a version store can be alternatives.
|
||||
|
||||
## Explicit Reconciliation
|
||||
|
||||
Expose the unresolved versions and provenance, then let an authorized person compare, choose or combine them and commit the new decision. Where retention permits, preserve recovery material until that commit; revocation policy can require earlier deletion and limit the available comparison. Provide perceivable conflict state, keyboard comparison and recoverable focus as explicit requirements. Another device may resolve concurrently, so the new revision still needs divergence handling. A revision store can provide alternatives without separate conflict files.
|
||||
|
||||
Connections are conditional: unfinished proposals support recovery after stale rejection; operation evidence informs refreshing observations; recoverable local commits enable resumable propagation; divergence can require preserved alternatives and an explicit resolution producing another local commit. No universal ordering or full-pattern recurrence is established. Documentary, pinned-source and authored-runtime mechanisms do not certify a complete deployed architecture or independent usefulness.
|
||||
|
||||
SCENARIO
|
||||
An offline drawing tool reports that a scene and pending upload queue survived one ordinary process restart. It also keeps a scene snapshot and conflict copy on the same laptop. The laptop may be destroyed. A scheduled off-device archive exists, but its last successful contents, key availability, deletion retention and restoration of pending operation identities have never been checked. The team asks whether synchronization plus these copies guarantees device-loss recovery.
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
Review the proposed frontend decision. Use the supplied reference only where its conditions hold. Return at most 220 words in four short bullets: (1) operation and visible state, (2) information and policy boundaries, (3) recovery action and rejected alternative, (4) missing contract. Explain reasoning without naming or quoting reference headings, candidate names, identifiers or presentation style. Do not infer undocumented guarantees. No tools, browsing, scores or tables. This is advice, not a claim that you implemented or tested anything.
|
||||
|
||||
REFERENCE
|
||||
Research reference: all mechanisms are draft hypotheses with limited scoped evidence. Apply only where their conditions hold; do not infer backend, storage, access, retention or backup guarantees.
|
||||
|
||||
## Local Draft State
|
||||
|
||||
Keep the base observation, unfinished proposal and current authoritative observation separate. A refresh, old receipt or rejected attempt must not erase newer typing. Decide navigation/restart lifetimes explicitly. Preservation, continued access, retention and deletion are conditional on access policy; recoverability is not permission to retain restricted content. Export needs a separate permission. A missing policy is unknown, not an unconditional retention or deletion rule.
|
||||
|
||||
## Version-Checked Mutation
|
||||
|
||||
When a proposal depends on an observed base, require the authority to check the expected version atomically with validation and commit. A stale rejection creates a review decision; do not blindly substitute a new version and resend. Preserve the proposal where policy permits and compare current state. A version precondition prevents certain stale writes; an operation identity and its retention contract address retry duplication. Neither substitutes for the other.
|
||||
|
||||
## Mutation Outcome Tracking
|
||||
|
||||
Track each operation as pending, accepted, explicitly rejected or unknown using operation-specific evidence. A timeout, cancelled browser wait or generic transport error does not prove rejection. A matching authenticated receipt can establish acceptance; weaker later errors cannot reverse it or clear newer work. Lookup and replay need the original immutable payload/identity, authority scope and documented retention window. Missing identity, expired receipts or ambiguous lookup absence require reconciliation, not a guaranteed fresh-ID retry. Stopping local tracking does not undo acceptance. Remote cancellation has domain preconditions and races; reversal of an accepted effect is a separate authorized operation whose outcome must also be tracked.
|
||||
|
||||
## Invalidate and Reproject
|
||||
|
||||
After acceptance, refresh affected observations while keeping unfinished work separate. Acceptance and view freshness are different states: a failed refresh leaves an accepted operation with unresolved observations, not a reason to replay the write. Guard obsolete read effects, but local request ordering cannot order opaque server versions or repair a replica consistency contract. Verify read-after-write/server ordering before claiming currentness. An authoritative result or suitable subscription may avoid a separate fetch.
|
||||
|
||||
## Local Commit Boundary
|
||||
|
||||
For a declared restart-safe offline save, commit the revision and pending propagation intent together under a named recoverable storage contract, then acknowledge saved locally. Storage failure leaves an unacknowledged save. Local acknowledgement is not remote acceptance. Ordinary process restart, interrupted commit, power loss, corruption and device loss need distinct evidence. Untested recovery is unknown rather than impossible. Same-device copies and synchronization do not establish independent backup. Define retention/failure scope and test clean-store restoration, including pending identities and reconciliation, before promising off-device recovery.
|
||||
|
||||
## Pending Synchronization
|
||||
|
||||
Persist stable revision and operation identities with pending work, distinguishing queued, sent and acknowledged states. Receiver acceptance does not prove every other device has received it or that an independent backup exists. Define deduplication, authorization and acknowledgement scope before replay. When resolution replaces operation P with revision c and operation Q, commit c, Q and P's superseded/non-retryable status in one recoverable transaction. After restart, P must not remain active. Local retirement cannot recall already transmitted P; receiver preconditions or reconciliation remain necessary. A late receipt for P updates only P's history, not Q or c.
|
||||
|
||||
## Conflict Copy
|
||||
|
||||
When divergence is detected and policy permits preservation, retain alternate content with distinct identity and provenance before destructive replacement. A selected working version is not a completed semantic resolution. Access, retention and export remain separate decisions, including after revocation. Copies may fail through missed detection, inaccessible notification, interrupted preservation or shared device failure; they are not a backup or universal atomicity guarantee. Domain-aware merge, rejection or a version store can be alternatives.
|
||||
|
||||
## Explicit Reconciliation
|
||||
|
||||
Expose the unresolved versions and provenance, then let an authorized person compare, choose or combine them and commit the new decision. Where retention permits, preserve recovery material until that commit; revocation policy can require earlier deletion and limit the available comparison. Provide perceivable conflict state, keyboard comparison and recoverable focus as explicit requirements. Another device may resolve concurrently, so the new revision still needs divergence handling. A revision store can provide alternatives without separate conflict files.
|
||||
|
||||
Connections are conditional: unfinished proposals support recovery after stale rejection; operation evidence informs refreshing observations; recoverable local commits enable resumable propagation; divergence can require preserved alternatives and an explicit resolution producing another local commit. No universal ordering or full-pattern recurrence is established. Documentary, pinned-source and authored-runtime mechanisms do not certify a complete deployed architecture or independent usefulness.
|
||||
|
||||
SCENARIO
|
||||
A report editor retains the original payload and operation identity K after an upload response was lost. The receiver guarantees deduplication for K only for 16 hours. Twenty-five hours later, lookup returns no record, but the service does not say whether no record means never accepted or expired history. The editor can resend K or create a fresh identity. The user asks for a duplicate-free automatic recovery plan.
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
Review the proposed frontend decision. Use the supplied reference only where its conditions hold. Return at most 220 words in four short bullets: (1) operation and visible state, (2) information and policy boundaries, (3) recovery action and rejected alternative, (4) missing contract. Explain reasoning without naming or quoting reference headings, candidate names, identifiers or presentation style. Do not infer undocumented guarantees. No tools, browsing, scores or tables. This is advice, not a claim that you implemented or tested anything.
|
||||
|
||||
REFERENCE
|
||||
Research reference: all mechanisms are draft hypotheses with limited scoped evidence. Apply only where their conditions hold; do not infer backend, storage, access, retention or backup guarantees.
|
||||
|
||||
## Decision 1
|
||||
|
||||
Keep the base observation, unfinished proposal and current authoritative observation separate. A refresh, old receipt or rejected attempt must not erase newer typing. Decide navigation/restart lifetimes explicitly. Preservation, continued access, retention and deletion are conditional on access policy; recoverability is not permission to retain restricted content. Export needs a separate permission. A missing policy is unknown, not an unconditional retention or deletion rule.
|
||||
|
||||
## Decision 2
|
||||
|
||||
When a proposal depends on an observed base, require the authority to check the expected version atomically with validation and commit. A stale rejection creates a review decision; do not blindly substitute a new version and resend. Preserve the proposal where policy permits and compare current state. A version precondition prevents certain stale writes; an operation identity and its retention contract address retry duplication. Neither substitutes for the other.
|
||||
|
||||
## Decision 3
|
||||
|
||||
Track each operation as pending, accepted, explicitly rejected or unknown using operation-specific evidence. A timeout, cancelled browser wait or generic transport error does not prove rejection. A matching authenticated receipt can establish acceptance; weaker later errors cannot reverse it or clear newer work. Lookup and replay need the original immutable payload/identity, authority scope and documented retention window. Missing identity, expired receipts or ambiguous lookup absence require reconciliation, not a guaranteed fresh-ID retry. Stopping local tracking does not undo acceptance. Remote cancellation has domain preconditions and races; reversal of an accepted effect is a separate authorized operation whose outcome must also be tracked.
|
||||
|
||||
## Decision 4
|
||||
|
||||
After acceptance, refresh affected observations while keeping unfinished work separate. Acceptance and view freshness are different states: a failed refresh leaves an accepted operation with unresolved observations, not a reason to replay the write. Guard obsolete read effects, but local request ordering cannot order opaque server versions or repair a replica consistency contract. Verify read-after-write/server ordering before claiming currentness. An authoritative result or suitable subscription may avoid a separate fetch.
|
||||
|
||||
## Decision 5
|
||||
|
||||
For a declared restart-safe offline save, commit the revision and pending propagation intent together under a named recoverable storage contract, then acknowledge saved locally. Storage failure leaves an unacknowledged save. Local acknowledgement is not remote acceptance. Ordinary process restart, interrupted commit, power loss, corruption and device loss need distinct evidence. Untested recovery is unknown rather than impossible. Same-device copies and synchronization do not establish independent backup. Define retention/failure scope and test clean-store restoration, including pending identities and reconciliation, before promising off-device recovery.
|
||||
|
||||
## Decision 6
|
||||
|
||||
Persist stable revision and operation identities with pending work, distinguishing queued, sent and acknowledged states. Receiver acceptance does not prove every other device has received it or that an independent backup exists. Define deduplication, authorization and acknowledgement scope before replay. When resolution replaces operation P with revision c and operation Q, commit c, Q and P's superseded/non-retryable status in one recoverable transaction. After restart, P must not remain active. Local retirement cannot recall already transmitted P; receiver preconditions or reconciliation remain necessary. A late receipt for P updates only P's history, not Q or c.
|
||||
|
||||
## Decision 7
|
||||
|
||||
When divergence is detected and policy permits preservation, retain alternate content with distinct identity and provenance before destructive replacement. A selected working version is not a completed semantic resolution. Access, retention and export remain separate decisions, including after revocation. Copies may fail through missed detection, inaccessible notification, interrupted preservation or shared device failure; they are not a backup or universal atomicity guarantee. Domain-aware merge, rejection or a version store can be alternatives.
|
||||
|
||||
## Decision 8
|
||||
|
||||
Expose the unresolved versions and provenance, then let an authorized person compare, choose or combine them and commit the new decision. Where retention permits, preserve recovery material until that commit; revocation policy can require earlier deletion and limit the available comparison. Provide perceivable conflict state, keyboard comparison and recoverable focus as explicit requirements. Another device may resolve concurrently, so the new revision still needs divergence handling. A revision store can provide alternatives without separate conflict files.
|
||||
|
||||
Connections are conditional: unfinished proposals support recovery after stale rejection; operation evidence informs refreshing observations; recoverable local commits enable resumable propagation; divergence can require preserved alternatives and an explicit resolution producing another local commit. No universal ordering or full-pattern recurrence is established. Documentary, pinned-source and authored-runtime mechanisms do not certify a complete deployed architecture or independent usefulness.
|
||||
|
||||
SCENARIO
|
||||
A report editor retains the original payload and operation identity K after an upload response was lost. The receiver guarantees deduplication for K only for 16 hours. Twenty-five hours later, lookup returns no record, but the service does not say whether no record means never accepted or expired history. The editor can resend K or create a fresh identity. The user asks for a duplicate-free automatic recovery plan.
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
Review the proposed frontend decision. Use the supplied reference only where its conditions hold. Return at most 220 words in four short bullets: (1) operation and visible state, (2) information and policy boundaries, (3) recovery action and rejected alternative, (4) missing contract. Explain reasoning without naming or quoting reference headings, candidate names, identifiers or presentation style. Do not infer undocumented guarantees. No tools, browsing, scores or tables. This is advice, not a claim that you implemented or tested anything.
|
||||
|
||||
REFERENCE
|
||||
Research reference: all mechanisms are draft hypotheses with limited scoped evidence. Apply only where their conditions hold; do not infer backend, storage, access, retention or backup guarantees.
|
||||
|
||||
## Decision 1
|
||||
|
||||
Keep the base observation, unfinished proposal and current authoritative observation separate. A refresh, old receipt or rejected attempt must not erase newer typing. Decide navigation/restart lifetimes explicitly. Preservation, continued access, retention and deletion are conditional on access policy; recoverability is not permission to retain restricted content. Export needs a separate permission. A missing policy is unknown, not an unconditional retention or deletion rule.
|
||||
|
||||
## Decision 2
|
||||
|
||||
When a proposal depends on an observed base, require the authority to check the expected version atomically with validation and commit. A stale rejection creates a review decision; do not blindly substitute a new version and resend. Preserve the proposal where policy permits and compare current state. A version precondition prevents certain stale writes; an operation identity and its retention contract address retry duplication. Neither substitutes for the other.
|
||||
|
||||
## Decision 3
|
||||
|
||||
Track each operation as pending, accepted, explicitly rejected or unknown using operation-specific evidence. A timeout, cancelled browser wait or generic transport error does not prove rejection. A matching authenticated receipt can establish acceptance; weaker later errors cannot reverse it or clear newer work. Lookup and replay need the original immutable payload/identity, authority scope and documented retention window. Missing identity, expired receipts or ambiguous lookup absence require reconciliation, not a guaranteed fresh-ID retry. Stopping local tracking does not undo acceptance. Remote cancellation has domain preconditions and races; reversal of an accepted effect is a separate authorized operation whose outcome must also be tracked.
|
||||
|
||||
## Decision 4
|
||||
|
||||
After acceptance, refresh affected observations while keeping unfinished work separate. Acceptance and view freshness are different states: a failed refresh leaves an accepted operation with unresolved observations, not a reason to replay the write. Guard obsolete read effects, but local request ordering cannot order opaque server versions or repair a replica consistency contract. Verify read-after-write/server ordering before claiming currentness. An authoritative result or suitable subscription may avoid a separate fetch.
|
||||
|
||||
## Decision 5
|
||||
|
||||
For a declared restart-safe offline save, commit the revision and pending propagation intent together under a named recoverable storage contract, then acknowledge saved locally. Storage failure leaves an unacknowledged save. Local acknowledgement is not remote acceptance. Ordinary process restart, interrupted commit, power loss, corruption and device loss need distinct evidence. Untested recovery is unknown rather than impossible. Same-device copies and synchronization do not establish independent backup. Define retention/failure scope and test clean-store restoration, including pending identities and reconciliation, before promising off-device recovery.
|
||||
|
||||
## Decision 6
|
||||
|
||||
Persist stable revision and operation identities with pending work, distinguishing queued, sent and acknowledged states. Receiver acceptance does not prove every other device has received it or that an independent backup exists. Define deduplication, authorization and acknowledgement scope before replay. When resolution replaces operation P with revision c and operation Q, commit c, Q and P's superseded/non-retryable status in one recoverable transaction. After restart, P must not remain active. Local retirement cannot recall already transmitted P; receiver preconditions or reconciliation remain necessary. A late receipt for P updates only P's history, not Q or c.
|
||||
|
||||
## Decision 7
|
||||
|
||||
When divergence is detected and policy permits preservation, retain alternate content with distinct identity and provenance before destructive replacement. A selected working version is not a completed semantic resolution. Access, retention and export remain separate decisions, including after revocation. Copies may fail through missed detection, inaccessible notification, interrupted preservation or shared device failure; they are not a backup or universal atomicity guarantee. Domain-aware merge, rejection or a version store can be alternatives.
|
||||
|
||||
## Decision 8
|
||||
|
||||
Expose the unresolved versions and provenance, then let an authorized person compare, choose or combine them and commit the new decision. Where retention permits, preserve recovery material until that commit; revocation policy can require earlier deletion and limit the available comparison. Provide perceivable conflict state, keyboard comparison and recoverable focus as explicit requirements. Another device may resolve concurrently, so the new revision still needs divergence handling. A revision store can provide alternatives without separate conflict files.
|
||||
|
||||
Connections are conditional: unfinished proposals support recovery after stale rejection; operation evidence informs refreshing observations; recoverable local commits enable resumable propagation; divergence can require preserved alternatives and an explicit resolution producing another local commit. No universal ordering or full-pattern recurrence is established. Documentary, pinned-source and authored-runtime mechanisms do not certify a complete deployed architecture or independent usefulness.
|
||||
|
||||
SCENARIO
|
||||
A project board receives an authenticated operation-specific receipt that batch import I was accepted. Refreshing the visible cards fails. The user is already editing a new card draft D. A background cache read later returns different opaque version labels; no ordering or read-after-write contract is supplied. A teammate proposes rerunning I and replacing the editor from that cache. Recommend the correct recovery boundaries.
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
Review the proposed frontend decision. Use the supplied reference only where its conditions hold. Return at most 220 words in four short bullets: (1) operation and visible state, (2) information and policy boundaries, (3) recovery action and rejected alternative, (4) missing contract. Explain reasoning without naming or quoting reference headings, candidate names, identifiers or presentation style. Do not infer undocumented guarantees. No tools, browsing, scores or tables. This is advice, not a claim that you implemented or tested anything.
|
||||
|
||||
REFERENCE
|
||||
Research reference: all mechanisms are draft hypotheses with limited scoped evidence. Apply only where their conditions hold; do not infer backend, storage, access, retention or backup guarantees.
|
||||
|
||||
## Local Draft State
|
||||
|
||||
Keep the base observation, unfinished proposal and current authoritative observation separate. A refresh, old receipt or rejected attempt must not erase newer typing. Decide navigation/restart lifetimes explicitly. Preservation, continued access, retention and deletion are conditional on access policy; recoverability is not permission to retain restricted content. Export needs a separate permission. A missing policy is unknown, not an unconditional retention or deletion rule.
|
||||
|
||||
## Version-Checked Mutation
|
||||
|
||||
When a proposal depends on an observed base, require the authority to check the expected version atomically with validation and commit. A stale rejection creates a review decision; do not blindly substitute a new version and resend. Preserve the proposal where policy permits and compare current state. A version precondition prevents certain stale writes; an operation identity and its retention contract address retry duplication. Neither substitutes for the other.
|
||||
|
||||
## Mutation Outcome Tracking
|
||||
|
||||
Track each operation as pending, accepted, explicitly rejected or unknown using operation-specific evidence. A timeout, cancelled browser wait or generic transport error does not prove rejection. A matching authenticated receipt can establish acceptance; weaker later errors cannot reverse it or clear newer work. Lookup and replay need the original immutable payload/identity, authority scope and documented retention window. Missing identity, expired receipts or ambiguous lookup absence require reconciliation, not a guaranteed fresh-ID retry. Stopping local tracking does not undo acceptance. Remote cancellation has domain preconditions and races; reversal of an accepted effect is a separate authorized operation whose outcome must also be tracked.
|
||||
|
||||
## Invalidate and Reproject
|
||||
|
||||
After acceptance, refresh affected observations while keeping unfinished work separate. Acceptance and view freshness are different states: a failed refresh leaves an accepted operation with unresolved observations, not a reason to replay the write. Guard obsolete read effects, but local request ordering cannot order opaque server versions or repair a replica consistency contract. Verify read-after-write/server ordering before claiming currentness. An authoritative result or suitable subscription may avoid a separate fetch.
|
||||
|
||||
## Local Commit Boundary
|
||||
|
||||
For a declared restart-safe offline save, commit the revision and pending propagation intent together under a named recoverable storage contract, then acknowledge saved locally. Storage failure leaves an unacknowledged save. Local acknowledgement is not remote acceptance. Ordinary process restart, interrupted commit, power loss, corruption and device loss need distinct evidence. Untested recovery is unknown rather than impossible. Same-device copies and synchronization do not establish independent backup. Define retention/failure scope and test clean-store restoration, including pending identities and reconciliation, before promising off-device recovery.
|
||||
|
||||
## Pending Synchronization
|
||||
|
||||
Persist stable revision and operation identities with pending work, distinguishing queued, sent and acknowledged states. Receiver acceptance does not prove every other device has received it or that an independent backup exists. Define deduplication, authorization and acknowledgement scope before replay. When resolution replaces operation P with revision c and operation Q, commit c, Q and P's superseded/non-retryable status in one recoverable transaction. After restart, P must not remain active. Local retirement cannot recall already transmitted P; receiver preconditions or reconciliation remain necessary. A late receipt for P updates only P's history, not Q or c.
|
||||
|
||||
## Conflict Copy
|
||||
|
||||
When divergence is detected and policy permits preservation, retain alternate content with distinct identity and provenance before destructive replacement. A selected working version is not a completed semantic resolution. Access, retention and export remain separate decisions, including after revocation. Copies may fail through missed detection, inaccessible notification, interrupted preservation or shared device failure; they are not a backup or universal atomicity guarantee. Domain-aware merge, rejection or a version store can be alternatives.
|
||||
|
||||
## Explicit Reconciliation
|
||||
|
||||
Expose the unresolved versions and provenance, then let an authorized person compare, choose or combine them and commit the new decision. Where retention permits, preserve recovery material until that commit; revocation policy can require earlier deletion and limit the available comparison. Provide perceivable conflict state, keyboard comparison and recoverable focus as explicit requirements. Another device may resolve concurrently, so the new revision still needs divergence handling. A revision store can provide alternatives without separate conflict files.
|
||||
|
||||
Connections are conditional: unfinished proposals support recovery after stale rejection; operation evidence informs refreshing observations; recoverable local commits enable resumable propagation; divergence can require preserved alternatives and an explicit resolution producing another local commit. No universal ordering or full-pattern recurrence is established. Documentary, pinned-source and authored-runtime mechanisms do not certify a complete deployed architecture or independent usefulness.
|
||||
|
||||
SCENARIO
|
||||
A project board receives an authenticated operation-specific receipt that batch import I was accepted. Refreshing the visible cards fails. The user is already editing a new card draft D. A background cache read later returns different opaque version labels; no ordering or read-after-write contract is supplied. A teammate proposes rerunning I and replacing the editor from that cache. Recommend the correct recovery boundaries.
|
||||
217
research/evaluations/language-replication-02/scenarios.json
Normal file
217
research/evaluations/language-replication-02/scenarios.json
Normal file
|
|
@ -0,0 +1,217 @@
|
|||
[
|
||||
{
|
||||
"id": "R02-S01",
|
||||
"title": "Support ticket attachments after access withdrawal",
|
||||
"brief": "A help-desk agent edits a confidential incident ticket offline. The application acknowledged a local save of the text and attachments. On reconnect, the authority explicitly rejects upload because the agent lost access. A teammate suggests leaving the editor open forever and exporting a recovery archive. The only stated product rule is that rejected edits should be recoverable; there is no rule about local viewing, retention, export or deletion after access withdrawal. Recommend the next states and actions.",
|
||||
"criteria": [
|
||||
{
|
||||
"id": "A1",
|
||||
"requirement": "Distinguish the acknowledged local save from explicitly denied remote upload."
|
||||
},
|
||||
{
|
||||
"id": "A2",
|
||||
"requirement": "Stop the unauthorized upload instead of retrying it as an unchanged permitted operation."
|
||||
},
|
||||
{
|
||||
"id": "A3",
|
||||
"requirement": "Identify the missing local retention policy after access withdrawal."
|
||||
},
|
||||
{
|
||||
"id": "A4",
|
||||
"requirement": "Make continued local viewing conditional on its access policy rather than promise it unconditionally."
|
||||
},
|
||||
{
|
||||
"id": "A5",
|
||||
"requirement": "Make exporting the attachment archive conditional on separate export permission."
|
||||
},
|
||||
{
|
||||
"id": "A6",
|
||||
"requirement": "Avoid unconditional destructive deletion while its policy is unspecified."
|
||||
}
|
||||
],
|
||||
"prohibited": [
|
||||
"Unconditionally retain or export confidential content after access withdrawal.",
|
||||
"Treat upload denial as evidence the local save never happened.",
|
||||
"Silently wipe the saved work without the required policy."
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "R02-S02",
|
||||
"title": "Accepted order and an uncertain reversal",
|
||||
"brief": "An order editor has an authenticated receipt that order submission A was accepted. The user has since typed an unsubmitted replacement B. Clicking Stop hides the spinner and cancels the browser wait. The user then requests reversal C through a separate endpoint; its response is lost. The service has not documented C's lookup, replay scope, retention or authorization semantics. Explain the statuses and recovery choices without inventing a successful reversal.",
|
||||
"criteria": [
|
||||
{
|
||||
"id": "A1",
|
||||
"requirement": "Keep submission A accepted on the strength of its matching authenticated receipt."
|
||||
},
|
||||
{
|
||||
"id": "A2",
|
||||
"requirement": "Preserve replacement draft B independently of A and C."
|
||||
},
|
||||
{
|
||||
"id": "A3",
|
||||
"requirement": "Do not treat cancelling the local browser wait as undoing A."
|
||||
},
|
||||
{
|
||||
"id": "A4",
|
||||
"requirement": "Recognize that reversal C has its own authorization preconditions."
|
||||
},
|
||||
{
|
||||
"id": "A5",
|
||||
"requirement": "Keep C's result unknown after the response is lost."
|
||||
},
|
||||
{
|
||||
"id": "A6",
|
||||
"requirement": "Identify C's missing lookup or bounded replay contract before asserting a safe retry."
|
||||
}
|
||||
],
|
||||
"prohibited": [
|
||||
"Claim Stop undid accepted A.",
|
||||
"Report C as successful solely because the user requested reversal.",
|
||||
"Resubmit B or C automatically with a fresh identity as guaranteed recovery."
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "R02-S03",
|
||||
"title": "Reconciliation committed in a split queue transaction",
|
||||
"brief": "A mobile checklist has pending upload P for revision a. Another device supplies incompatible revision b. A user compares both and records chosen revision c. The implementation saves c and queues new operation Q in one transaction, but marks P non-retryable in a later transaction. A process can die between the transactions, and an already transmitted P may still complete. The receiver's version/ancestry enforcement is undocumented. Specify the minimal correction and receipt handling.",
|
||||
"criteria": [
|
||||
{
|
||||
"id": "A1",
|
||||
"requirement": "Include P's superseded/non-retryable status in the same recoverable commit as c and Q."
|
||||
},
|
||||
{
|
||||
"id": "A2",
|
||||
"requirement": "Do not acknowledge the resolution before the required combined commit succeeds."
|
||||
},
|
||||
{
|
||||
"id": "A3",
|
||||
"requirement": "Retain the identities and provenance of the compared inputs for the resolution."
|
||||
},
|
||||
{
|
||||
"id": "A4",
|
||||
"requirement": "Do not claim local retirement of P recalls its already transmitted request."
|
||||
},
|
||||
{
|
||||
"id": "A5",
|
||||
"requirement": "Identify the missing receiver version/ancestry contract before claiming the stale write cannot apply."
|
||||
},
|
||||
{
|
||||
"id": "A6",
|
||||
"requirement": "Apply a late receipt for P only to P's history rather than clearing Q or replacing c."
|
||||
}
|
||||
],
|
||||
"prohibited": [
|
||||
"Call the two local transactions atomic together.",
|
||||
"Declare P cancelled remotely because the local flag changed.",
|
||||
"Clear Q when P later acknowledges."
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "R02-S04",
|
||||
"title": "Snapshots, an off-device archive and missing restore proof",
|
||||
"brief": "An offline drawing tool reports that a scene and pending upload queue survived one ordinary process restart. It also keeps a scene snapshot and conflict copy on the same laptop. The laptop may be destroyed. A scheduled off-device archive exists, but its last successful contents, key availability, deletion retention and restoration of pending operation identities have never been checked. The team asks whether synchronization plus these copies guarantees device-loss recovery.",
|
||||
"criteria": [
|
||||
{
|
||||
"id": "A1",
|
||||
"requirement": "Distinguish the observed process-restart result from device-loss recovery."
|
||||
},
|
||||
{
|
||||
"id": "A2",
|
||||
"requirement": "Do not treat the copies on the same laptop as independent device-loss protection."
|
||||
},
|
||||
{
|
||||
"id": "A3",
|
||||
"requirement": "Do not claim that synchronization by itself is a separately recoverable backup."
|
||||
},
|
||||
{
|
||||
"id": "A4",
|
||||
"requirement": "Keep device-loss recovery unverified rather than declare it guaranteed or impossible."
|
||||
},
|
||||
{
|
||||
"id": "A5",
|
||||
"requirement": "Identify the missing restore-key availability contract for the independent archive."
|
||||
},
|
||||
{
|
||||
"id": "A6",
|
||||
"requirement": "Require a clean-store restore test of the saved pending operation identities before promising resume safety."
|
||||
}
|
||||
],
|
||||
"prohibited": [
|
||||
"Guarantee recovery because sync and two local copies exist.",
|
||||
"Declare recovery impossible merely because the archive was not tested.",
|
||||
"Automatically replay restored work with new identities."
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "R02-S05",
|
||||
"title": "Lookup absence after a bounded receipt expires",
|
||||
"brief": "A report editor retains the original payload and operation identity K after an upload response was lost. The receiver guarantees deduplication for K only for 16 hours. Twenty-five hours later, lookup returns no record, but the service does not say whether no record means never accepted or expired history. The editor can resend K or create a fresh identity. The user asks for a duplicate-free automatic recovery plan.",
|
||||
"criteria": [
|
||||
{
|
||||
"id": "A1",
|
||||
"requirement": "Keep the old operation's acceptance unknown despite the ambiguous lookup absence."
|
||||
},
|
||||
{
|
||||
"id": "A2",
|
||||
"requirement": "Recognize that the documented 16-hour deduplication guarantee no longer covers this replay."
|
||||
},
|
||||
{
|
||||
"id": "A3",
|
||||
"requirement": "Do not promise a duplicate-free resend of K outside that retention guarantee."
|
||||
},
|
||||
{
|
||||
"id": "A4",
|
||||
"requirement": "Do not claim a fresh identity makes the same logical submission duplicate-safe."
|
||||
},
|
||||
{
|
||||
"id": "A5",
|
||||
"requirement": "Retain the original payload and identity for reconciliation."
|
||||
},
|
||||
{
|
||||
"id": "A6",
|
||||
"requirement": "Identify the missing meaning of lookup absence after receipt history expires."
|
||||
}
|
||||
],
|
||||
"prohibited": [
|
||||
"Equate absent expired lookup history with explicit rejection.",
|
||||
"Guarantee a safe replay using either K or a new identity after the window."
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "R02-S06",
|
||||
"title": "Accepted import with incomplete projection and a newer edit",
|
||||
"brief": "A project board receives an authenticated operation-specific receipt that batch import I was accepted. Refreshing the visible cards fails. The user is already editing a new card draft D. A background cache read later returns different opaque version labels; no ordering or read-after-write contract is supplied. A teammate proposes rerunning I and replacing the editor from that cache. Recommend the correct recovery boundaries.",
|
||||
"criteria": [
|
||||
{
|
||||
"id": "A1",
|
||||
"requirement": "Keep import I accepted despite the failed view refresh."
|
||||
},
|
||||
{
|
||||
"id": "A2",
|
||||
"requirement": "Distinguish accepted operation state from unresolved view freshness."
|
||||
},
|
||||
{
|
||||
"id": "A3",
|
||||
"requirement": "Preserve the newer draft D when observations refresh."
|
||||
},
|
||||
{
|
||||
"id": "A4",
|
||||
"requirement": "Do not rerun the accepted import solely to repair its failed projection refresh."
|
||||
},
|
||||
{
|
||||
"id": "A5",
|
||||
"requirement": "Do not infer currentness by sorting or comparing opaque version labels."
|
||||
},
|
||||
{
|
||||
"id": "A6",
|
||||
"requirement": "Identify the missing read-after-write or server-ordering contract before asserting fresh cards."
|
||||
}
|
||||
],
|
||||
"prohibited": [
|
||||
"Retry the import because the view failed.",
|
||||
"Discard D on receipt or cache completion.",
|
||||
"Order opaque labels as proof the cache is fresh."
|
||||
]
|
||||
}
|
||||
]
|
||||
|
|
@ -0,0 +1,540 @@
|
|||
{
|
||||
"schema_version": 1,
|
||||
"reviewer": {
|
||||
"name": null,
|
||||
"role": null,
|
||||
"independent_human": false,
|
||||
"assignment_seen": null,
|
||||
"prior_scores_seen": null
|
||||
},
|
||||
"answers": {
|
||||
"R02-S01-U": {
|
||||
"available": null,
|
||||
"ratings": {
|
||||
"A1": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A2": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A3": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A4": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A5": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A6": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
}
|
||||
},
|
||||
"unsupported_claims": null,
|
||||
"ambiguity": [],
|
||||
"suspected_condition_clues": []
|
||||
},
|
||||
"R02-S01-V": {
|
||||
"available": null,
|
||||
"ratings": {
|
||||
"A1": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A2": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A3": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A4": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A5": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A6": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
}
|
||||
},
|
||||
"unsupported_claims": null,
|
||||
"ambiguity": [],
|
||||
"suspected_condition_clues": []
|
||||
},
|
||||
"R02-S02-U": {
|
||||
"available": null,
|
||||
"ratings": {
|
||||
"A1": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A2": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A3": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A4": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A5": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A6": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
}
|
||||
},
|
||||
"unsupported_claims": null,
|
||||
"ambiguity": [],
|
||||
"suspected_condition_clues": []
|
||||
},
|
||||
"R02-S02-V": {
|
||||
"available": null,
|
||||
"ratings": {
|
||||
"A1": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A2": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A3": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A4": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A5": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A6": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
}
|
||||
},
|
||||
"unsupported_claims": null,
|
||||
"ambiguity": [],
|
||||
"suspected_condition_clues": []
|
||||
},
|
||||
"R02-S03-U": {
|
||||
"available": null,
|
||||
"ratings": {
|
||||
"A1": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A2": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A3": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A4": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A5": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A6": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
}
|
||||
},
|
||||
"unsupported_claims": null,
|
||||
"ambiguity": [],
|
||||
"suspected_condition_clues": []
|
||||
},
|
||||
"R02-S03-V": {
|
||||
"available": null,
|
||||
"ratings": {
|
||||
"A1": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A2": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A3": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A4": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A5": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A6": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
}
|
||||
},
|
||||
"unsupported_claims": null,
|
||||
"ambiguity": [],
|
||||
"suspected_condition_clues": []
|
||||
},
|
||||
"R02-S04-U": {
|
||||
"available": null,
|
||||
"ratings": {
|
||||
"A1": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A2": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A3": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A4": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A5": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A6": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
}
|
||||
},
|
||||
"unsupported_claims": null,
|
||||
"ambiguity": [],
|
||||
"suspected_condition_clues": []
|
||||
},
|
||||
"R02-S04-V": {
|
||||
"available": null,
|
||||
"ratings": {
|
||||
"A1": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A2": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A3": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A4": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A5": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A6": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
}
|
||||
},
|
||||
"unsupported_claims": null,
|
||||
"ambiguity": [],
|
||||
"suspected_condition_clues": []
|
||||
},
|
||||
"R02-S05-U": {
|
||||
"available": null,
|
||||
"ratings": {
|
||||
"A1": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A2": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A3": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A4": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A5": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A6": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
}
|
||||
},
|
||||
"unsupported_claims": null,
|
||||
"ambiguity": [],
|
||||
"suspected_condition_clues": []
|
||||
},
|
||||
"R02-S05-V": {
|
||||
"available": null,
|
||||
"ratings": {
|
||||
"A1": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A2": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A3": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A4": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A5": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A6": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
}
|
||||
},
|
||||
"unsupported_claims": null,
|
||||
"ambiguity": [],
|
||||
"suspected_condition_clues": []
|
||||
},
|
||||
"R02-S06-U": {
|
||||
"available": null,
|
||||
"ratings": {
|
||||
"A1": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A2": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A3": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A4": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A5": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A6": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
}
|
||||
},
|
||||
"unsupported_claims": null,
|
||||
"ambiguity": [],
|
||||
"suspected_condition_clues": []
|
||||
},
|
||||
"R02-S06-V": {
|
||||
"available": null,
|
||||
"ratings": {
|
||||
"A1": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A2": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A3": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A4": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A5": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
},
|
||||
"A6": {
|
||||
"value": null,
|
||||
"quote": null,
|
||||
"locator": null,
|
||||
"reason": null
|
||||
}
|
||||
},
|
||||
"unsupported_claims": null,
|
||||
"ambiguity": [],
|
||||
"suspected_condition_clues": []
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
# LEARN-0019 — Fresh atomic-criteria presentation replication
|
||||
|
||||
Status: preregistered before inference. Owner/reviewer: Codex, author-model.
|
||||
Date: 2026-10-04. Work owner: FEP-WP-0009 T02.
|
||||
Stream: cross-stream. New comparison group: presentation-atomic-replication-v2.
|
||||
This protocol differs from the consumed retrospective pilot and from static/runtime
|
||||
mechanism studies; it cannot be pooled into their comparison windows.
|
||||
|
||||
Question: does named organization change constraint/recovery recommendations versus
|
||||
the same guidance under numbered headings on fresh authored variants? Both conditions
|
||||
share eight exact bodies and caveats. Previous pilot-01 model-review adjudication was
|
||||
18/24 in each condition; that supplies no naming benefit or equivalence claim.
|
||||
Material disconfirmation of an assumed benefit includes tied/mixed criterion outcomes,
|
||||
additional unsupported guarantees or common failures despite shared guidance.
|
||||
|
||||
Execute the [frozen replication packet](../../evaluations/language-replication-02/protocol.md)
|
||||
with 12 isolated llm-connect calls, requested 1,400-output-token caps and no retries,
|
||||
as explicitly authorized by the user. Six pairs have six atomic criteria each,
|
||||
not the old pilot's four. Commit inputs/runner/prompts before execution. Retain
|
||||
failed generations and exclude their pairs from quality comparison. Score masked
|
||||
outputs against the exact rubric and commit initial author-model judgments before
|
||||
unmasking. Exposure and uncertainty stay explicit; human validation remains separate.
|
||||
|
||||
Acceptance: a retained bounded replication, descriptive per-criterion/scenario
|
||||
comparison and supported editorial conclusion. No M1 pass, independent human evidence,
|
||||
production applicability or general efficacy estimate follows. A consistent descriptive
|
||||
advantage warrants independent follow-up; tied/mixed outcomes retain hypotheses and
|
||||
are inconclusive. Shared errors may motivate a separate correction, not post-hoc rubric
|
||||
changes or selected reruns. One question counts one outcome, not 72 criterion ratings.
|
||||
|
||||
Active research time is unknown: no timer was started before preparation. Do not infer
|
||||
it from the preregistration commit, chat duration, tokens or check subprocesses. Sum
|
||||
measured request durations as execution minutes separately; use token/cost values
|
||||
only where actual receipts establish them. No extra model review is within this grant.
|
||||
|
||||
Outcome is recorded after the authorized run in the replication review and LEARN-0019
|
||||
ledger entry. WP-0009 T01 retains independent human pilot-01 ratings; T03 retains
|
||||
M1 re-assessment. WP-0008 retains actual consumer decision/outcome evidence. A new
|
||||
comparison group does not close WP-0006's comparable-window requirement.
|
||||
308
scripts/language_replication.py
Normal file
308
scripts/language_replication.py
Normal file
|
|
@ -0,0 +1,308 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Freeze, verify and execute the bounded fresh presentation replication."""
|
||||
import argparse
|
||||
from collections import Counter
|
||||
from datetime import datetime, timezone
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import secrets
|
||||
import subprocess
|
||||
import time
|
||||
from types import SimpleNamespace
|
||||
|
||||
from reference_consume import ROOT, published
|
||||
from reference_preview import require
|
||||
|
||||
PACKET = Path('research/evaluations/language-replication-02')
|
||||
INPUTS = ('protocol.md', 'instructions.txt', 'scenarios.json', 'guidance.json')
|
||||
DEPENDENCIES = ('scripts/language_replication.py', 'scripts/reference_consume.py',
|
||||
'scripts/reference_preview.py', 'scripts/classification_coverage.py', 'scripts/research_bench.py')
|
||||
|
||||
|
||||
def digest(raw):
|
||||
return hashlib.sha256(raw).hexdigest()
|
||||
|
||||
|
||||
def read(path):
|
||||
return json.loads(path.read_text())
|
||||
|
||||
|
||||
def write_new(path, value):
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
text = value if isinstance(value, str) else json.dumps(value, indent=2, allow_nan=False) + '\n'
|
||||
with path.open('x') as stream:
|
||||
stream.write(text)
|
||||
|
||||
|
||||
def scenarios_check(scenarios):
|
||||
require(len(scenarios) == 6 and len({s['id'] for s in scenarios}) == 6, 'Six unique scenarios required')
|
||||
for scenario in scenarios:
|
||||
require(scenario['brief'].strip() and scenario['title'].strip() and scenario['prohibited'], 'Incomplete scenario')
|
||||
require([c['id'] for c in scenario['criteria']] == [f'A{i}' for i in range(1, 7)], 'Six atomic criteria required')
|
||||
require(all(c['requirement'].strip() for c in scenario['criteria']), 'Empty criterion')
|
||||
|
||||
|
||||
def packs(guidance):
|
||||
result = {}
|
||||
for condition in ('patterns', 'prose'):
|
||||
parts = [guidance['intro']]
|
||||
for number, item in enumerate(guidance['items'], 1):
|
||||
heading = item['title'] if condition == 'patterns' else f'Decision {number}'
|
||||
parts.append(f'## {heading}\n\n{item["body"]}')
|
||||
parts.append(guidance['closing'])
|
||||
result[condition] = '\n\n'.join(parts) + '\n'
|
||||
return result
|
||||
|
||||
|
||||
def prompt(packet, assignment, scenario):
|
||||
return ((packet / 'instructions.txt').read_text() + '\nREFERENCE\n' +
|
||||
(packet / f'pack-{assignment["condition"]}.md').read_text() +
|
||||
'\nSCENARIO\n' + scenario['brief'] + '\n')
|
||||
|
||||
|
||||
def score_template(scenarios):
|
||||
return {'schema_version': 1, 'reviewer': {'name': None, 'role': None, 'independent_human': False,
|
||||
'assignment_seen': None, 'prior_scores_seen': None},
|
||||
'answers': {s['id'] + '-' + label: {
|
||||
'available': None,
|
||||
'ratings': {c['id']: {'value': None, 'quote': None, 'locator': None, 'reason': None}
|
||||
for c in s['criteria']},
|
||||
'unsupported_claims': None, 'ambiguity': [], 'suspected_condition_clues': []}
|
||||
for s in scenarios for label in ('U', 'V')}}
|
||||
|
||||
|
||||
def prepare(root=ROOT):
|
||||
packet = root / PACKET
|
||||
require(not (packet / 'freeze.json').exists(), 'Packet already frozen; do not overwrite')
|
||||
index, pin, manifest = published(root, '0.1')
|
||||
scenarios = read(packet / 'scenarios.json')
|
||||
scenarios_check(scenarios)
|
||||
old = read(root / 'research/evaluations/language-pilot-01/scenarios.json')
|
||||
require(not ({s['brief'] for s in scenarios} & {s['brief'] for s in old}), 'Consumed brief reused')
|
||||
guidance = read(packet / 'guidance.json')
|
||||
core = {c['id']: c for c in index['candidates'] if c['subset'] == 'core'}
|
||||
require(len(guidance['items']) == 8 and {i['candidate'] for i in guidance['items']} == set(core), 'Guidance must cover pinned core')
|
||||
for item in guidance['items']:
|
||||
require(item['title'] == core[item['candidate']]['title'] and item['body'].strip(), 'Guidance identity/body mismatch')
|
||||
pack_text = packs(guidance)
|
||||
for condition, text in pack_text.items():
|
||||
write_new(packet / f'pack-{condition}.md', text)
|
||||
equivalence = {'method': 'Only headings vary; all eight ordered bodies, introduction and closing are byte-identical.',
|
||||
'blocks': [{'candidate': i['candidate'], 'body_sha256': digest(i['body'].encode()),
|
||||
'source_path': core[i['candidate']]['path'], 'source_sha256': core[i['candidate']]['sha256']}
|
||||
for i in guidance['items']],
|
||||
'word_counts': {k: len(v.split()) for k, v in pack_text.items()}}
|
||||
write_new(packet / 'pack-equivalence.json', equivalence)
|
||||
assignments = []
|
||||
for number, scenario in enumerate(scenarios):
|
||||
conditions = ['patterns', 'prose'] if number % 2 == 0 else ['prose', 'patterns']
|
||||
labels = ['U', 'V'] if secrets.randbelow(2) else ['V', 'U']
|
||||
for condition, label in zip(conditions, labels):
|
||||
assignments.append({'run': f'r{len(assignments)+1:02d}', 'scenario': scenario['id'],
|
||||
'condition': condition, 'blind_id': scenario['id'] + '-' + label})
|
||||
write_new(packet / 'assignment.json', assignments)
|
||||
generated = ['pack-patterns.md', 'pack-prose.md', 'pack-equivalence.json', 'assignment.json', 'scores-template.json']
|
||||
write_new(packet / 'scores-template.json', score_template(scenarios))
|
||||
by_id = {s['id']: s for s in scenarios}
|
||||
for assignment in assignments:
|
||||
path = 'requests/' + assignment['run'] + '.txt'
|
||||
write_new(packet / path, prompt(packet, assignment, by_id[assignment['scenario']]))
|
||||
generated.append(path)
|
||||
paths = [PACKET / name for name in (*INPUTS, *generated)] + [Path(p) for p in DEPENDENCIES]
|
||||
write_new(packet / 'freeze.json', {
|
||||
'schema_version': 1, 'packet': 'language-replication-02',
|
||||
'prepared_at': datetime.now(timezone.utc).isoformat(),
|
||||
'reference': dict(pin, version='0.1', corpus_revision=manifest['corpus_revision']),
|
||||
'files': {str(p): digest((root / p).read_bytes()) for p in paths},
|
||||
'sessions': 12, 'max_output_tokens_per_call': 1400,
|
||||
'limits': ['Author-created fresh variants, not held-out M3 retrieval or independent validation.',
|
||||
'No inference at preparation; execution needs explicit user authorization.']})
|
||||
check(root)
|
||||
|
||||
|
||||
def check(root=ROOT):
|
||||
packet = root / PACKET
|
||||
frozen = read(packet / 'freeze.json')
|
||||
require(frozen['schema_version'] == 1 and frozen['sessions'] == 12 and frozen['max_output_tokens_per_call'] == 1400, 'Invalid freeze contract')
|
||||
require(set(DEPENDENCIES) <= set(frozen['files']), 'Missing frozen dependencies')
|
||||
for relative, expected in frozen['files'].items():
|
||||
path = root / relative
|
||||
require(path.resolve().is_relative_to(root.resolve()), 'Unsafe frozen path')
|
||||
require(digest(path.read_bytes()) == expected, 'Frozen input changed: ' + relative)
|
||||
_, pin, manifest = published(root, frozen['reference']['version'])
|
||||
require(frozen['reference'] == dict(pin, version=manifest['version'], corpus_revision=manifest['corpus_revision']), 'Reference pin drift')
|
||||
scenarios = read(packet / 'scenarios.json')
|
||||
scenarios_check(scenarios)
|
||||
guidance = read(packet / 'guidance.json')
|
||||
for condition, text in packs(guidance).items():
|
||||
require((packet / f'pack-{condition}.md').read_text() == text, 'Presentation body/heading mismatch')
|
||||
assignments = read(packet / 'assignment.json')
|
||||
require([a['run'] for a in assignments] == [f'r{i:02d}' for i in range(1, 13)], 'Invalid run identities')
|
||||
by_id = {s['id']: s for s in scenarios}
|
||||
for number, scenario in enumerate(scenarios):
|
||||
pair = assignments[2*number:2*number+2]
|
||||
require({a['scenario'] for a in pair} == {scenario['id']}, 'Invalid pairing')
|
||||
require({a['blind_id'] for a in pair} == {scenario['id'] + '-U', scenario['id'] + '-V'}, 'Invalid masking labels')
|
||||
require([a['condition'] for a in pair] == (['patterns', 'prose'] if number % 2 == 0 else ['prose', 'patterns']), 'Counterbalance changed')
|
||||
for assignment in assignments:
|
||||
path = packet / 'requests' / (assignment['run'] + '.txt')
|
||||
require(path.read_text() == prompt(packet, assignment, by_id[assignment['scenario']]), 'Prompt changed or scoring material supplied')
|
||||
require(str(path.relative_to(root)) in frozen['files'], 'Unfrozen prompt')
|
||||
require(read(packet / 'scores-template.json') == score_template(scenarios), 'Blank score template changed')
|
||||
return frozen, assignments, scenarios
|
||||
|
||||
|
||||
def committed(root, frozen):
|
||||
paths = [*frozen['files'], str(PACKET / 'freeze.json')]
|
||||
for path in paths:
|
||||
raw = subprocess.check_output(['git', 'show', 'HEAD:' + path], cwd=root, stderr=subprocess.DEVNULL)
|
||||
require(raw == (root / path).read_bytes(), 'Commit frozen packet before inference')
|
||||
|
||||
|
||||
def run(max_calls, root=ROOT, adapter_factory=None):
|
||||
frozen, assignments, _ = check(root)
|
||||
require(type(max_calls) is int and 0 < max_calls <= 12, 'Explicit ceiling of at most 12 calls required')
|
||||
committed(root, frozen)
|
||||
packet = root / PACKET
|
||||
pending = []
|
||||
for assignment in assignments:
|
||||
directory = packet / 'raw' / assignment['run']
|
||||
if directory.exists():
|
||||
started = read(directory / 'started.json')
|
||||
require(started['attempts_reserved'] == 1 and started['prompt_sha256'] == digest((packet / 'requests' / (assignment['run'] + '.txt')).read_bytes()), 'Invalid prior reservation')
|
||||
else:
|
||||
pending.append(assignment)
|
||||
used = len(assignments) - len(pending)
|
||||
require(used + len(pending) <= max_calls, 'Requested sessions exceed aggregate call ceiling')
|
||||
if adapter_factory is None:
|
||||
from research_bench import LLMConnectAdapter
|
||||
adapter_factory = lambda: LLMConnectAdapter('http://127.0.0.1:18081', 'custodian-triage-balanced', 1400, diagnostics=True)
|
||||
for assignment in pending:
|
||||
directory = packet / 'raw' / assignment['run']
|
||||
directory.mkdir(parents=True, exist_ok=False)
|
||||
text = (packet / 'requests' / (assignment['run'] + '.txt')).read_text()
|
||||
started = dict(assignment, attempts_reserved=1, prompt_sha256=digest(text.encode()),
|
||||
freeze_sha256=digest((packet / 'freeze.json').read_bytes()), max_output_tokens=1400)
|
||||
write_new(directory / 'started.json', started) # Reserve before any external call.
|
||||
adapter = adapter_factory()
|
||||
receipt = dict(started, endpoint=adapter.endpoint, config=adapter.config.to_dict())
|
||||
then = time.monotonic()
|
||||
try:
|
||||
result = adapter.generate(SimpleNamespace(prompt=text, stage_id='decision-review'))
|
||||
write_new(directory / 'answer.md', result.markdown)
|
||||
write_new(packet / 'masked' / (assignment['blind_id'] + '.md'), result.markdown)
|
||||
receipt.update(status='completed', output_sha256=digest(result.markdown.encode()), output_words=len(result.markdown.split()))
|
||||
except Exception as exc:
|
||||
receipt.update(status='failed', error_type=type(exc).__name__)
|
||||
if adapter.rejected_content:
|
||||
write_new(directory / 'rejected-response.md', adapter.rejected_content)
|
||||
receipt.update(elapsed_seconds=round(time.monotonic() - then, 3), responses=adapter.responses)
|
||||
write_new(directory / 'receipt.json', receipt)
|
||||
print(json.dumps({'run': assignment['run'], 'status': receipt['status']}), flush=True)
|
||||
|
||||
|
||||
def scores_check(sheet, scenarios, complete=False):
|
||||
expected = score_template(scenarios)
|
||||
require(sheet.get('schema_version') == 1 and set(sheet['answers']) == set(expected['answers']), 'Score answer inventory mismatch')
|
||||
reviewer = sheet['reviewer']
|
||||
require(type(reviewer['independent_human']) is bool, 'Invalid review independence declaration')
|
||||
require(not reviewer['independent_human'] or reviewer['role'] == 'human', 'Model review cannot be independent human validation')
|
||||
if complete:
|
||||
require(isinstance(reviewer['name'], str) and reviewer['name'].strip() and reviewer['role'] in {'author-model', 'independent-model', 'human'}, 'Reviewer provenance required')
|
||||
require(type(reviewer['assignment_seen']) is bool and type(reviewer['prior_scores_seen']) is bool, 'Exposure disclosure required')
|
||||
for ident, answer in sheet['answers'].items():
|
||||
require(answer['available'] is None or type(answer['available']) is bool, 'Invalid answer availability')
|
||||
require(set(answer['ratings']) == set(expected['answers'][ident]['ratings']), 'Atomic criterion inventory mismatch')
|
||||
if complete:
|
||||
require(type(answer['available']) is bool, 'Answer availability required')
|
||||
for rating in answer['ratings'].values():
|
||||
value = rating['value']
|
||||
require(value is None or type(value) is int and value in (0, 1), 'Atomic scores are binary or unreviewed null')
|
||||
require(all(rating[k] is None or isinstance(rating[k], str) and rating[k].strip()
|
||||
for k in ('quote', 'locator', 'reason')), 'Rating evidence must be text')
|
||||
require(answer['available'] is not False or value is None, 'Unavailable answer cannot be scored zero')
|
||||
if value is not None:
|
||||
require(rating['reason'] and (rating['quote'] or rating['locator']), 'Rating evidence and reason required')
|
||||
if complete and answer['available']:
|
||||
require(value is not None, 'Available answer has unreviewed criterion')
|
||||
claims = answer['unsupported_claims']
|
||||
require(claims is None or isinstance(claims, list) and all(c['quote'] and c['reason'] for c in claims), 'Unsupported claims need evidence')
|
||||
if complete and answer['available']:
|
||||
require(claims is not None, 'Unsupported-claim review required')
|
||||
require(all(isinstance(answer[k], list) and all(isinstance(v, str) for v in answer[k])
|
||||
for k in ('ambiguity', 'suspected_condition_clues')), 'Review caveats must be lists')
|
||||
return True
|
||||
|
||||
|
||||
def summarize(sheet, root=ROOT):
|
||||
frozen, assignments, scenarios = check(root)
|
||||
scores_check(sheet, scenarios, complete=True)
|
||||
totals = {c: Counter(completed_pairs=0, fulfilled=0, unsafe=0, criteria=0) for c in ('patterns', 'prose')}
|
||||
pairs = []
|
||||
for scenario in scenarios:
|
||||
pair = [a for a in assignments if a['scenario'] == scenario['id']]
|
||||
valid = True
|
||||
for assignment in pair:
|
||||
directory = root / PACKET / 'raw' / assignment['run']
|
||||
receipt_path = directory / 'receipt.json'
|
||||
rating = sheet['answers'][assignment['blind_id']]
|
||||
if not receipt_path.exists():
|
||||
valid = False
|
||||
require(not rating['available'], 'Unexecuted answer cannot be marked available')
|
||||
continue
|
||||
receipt = read(receipt_path)
|
||||
require(receipt['freeze_sha256'] == digest((root / PACKET / 'freeze.json').read_bytes()), 'Receipt freeze mismatch')
|
||||
available = receipt['status'] == 'completed'
|
||||
require(rating['available'] == available, 'Scoring availability differs from retained result')
|
||||
if available:
|
||||
raw = (directory / 'answer.md').read_bytes()
|
||||
require(digest(raw) == receipt['output_sha256'], 'Retained answer changed')
|
||||
require((root / PACKET / 'masked' / (assignment['blind_id'] + '.md')).read_bytes() == raw, 'Masked answer changed')
|
||||
for item in [*rating['ratings'].values(), *rating['unsupported_claims']]:
|
||||
if item.get('quote'):
|
||||
require(item['quote'] in raw.decode(), 'Quoted score evidence not in retained answer')
|
||||
else:
|
||||
valid = False
|
||||
results = {}
|
||||
if valid:
|
||||
for assignment in pair:
|
||||
rating = sheet['answers'][assignment['blind_id']]
|
||||
results[assignment['condition']] = {'fulfilled': sum(r['value'] for r in rating['ratings'].values()),
|
||||
'unsafe': len(rating['unsupported_claims'])}
|
||||
totals[assignment['condition']].update(completed_pairs=1, criteria=6, **results[assignment['condition']])
|
||||
pairs.append({'scenario': scenario['id'], 'included': valid, 'results': results})
|
||||
return {'schema_version': 1, 'reviewer': sheet['reviewer'], 'totals': {k: dict(v) for k, v in totals.items()},
|
||||
'pairs': pairs, 'limits': ['Descriptive author-created replication, not proof of superiority/equivalence.',
|
||||
'Reviewer role is declared provenance; no automatic independent-human or M1 claim.']}
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('command', choices=['prepare', 'check', 'run', 'scores-check', 'summarize'])
|
||||
parser.add_argument('--max-calls', type=int)
|
||||
parser.add_argument('--scores', type=Path)
|
||||
parser.add_argument('--complete', action='store_true')
|
||||
args = parser.parse_args()
|
||||
try:
|
||||
if args.command == 'prepare':
|
||||
prepare()
|
||||
elif args.command == 'run':
|
||||
run(args.max_calls)
|
||||
elif args.command in {'scores-check', 'summarize'}:
|
||||
require(args.scores, 'Score record required')
|
||||
_, _, scenarios = check()
|
||||
sheet = read(args.scores)
|
||||
if args.command == 'summarize':
|
||||
print(json.dumps(summarize(sheet), indent=2))
|
||||
return
|
||||
scores_check(sheet, scenarios, args.complete)
|
||||
else:
|
||||
check()
|
||||
print('Replication packet/scoring contract verified; no empirical gate inferred.')
|
||||
except (ValueError, TypeError, KeyError, OSError, subprocess.CalledProcessError) as exc:
|
||||
parser.exit(1, f'Replication failed: {exc}\n')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
157
tests/test_language_replication.py
Normal file
157
tests/test_language_replication.py
Normal file
|
|
@ -0,0 +1,157 @@
|
|||
"""Guard isolation, immutable budgets and evidence-bearing atomic scoring."""
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / 'scripts'))
|
||||
import language_replication as replication
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def packet(tmp_path):
|
||||
shutil.copytree(ROOT / 'reference', tmp_path / 'reference')
|
||||
(tmp_path / '.git').symlink_to(ROOT / '.git', target_is_directory=True)
|
||||
for name in (*replication.INPUTS,):
|
||||
target = tmp_path / replication.PACKET / name
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copy(ROOT / replication.PACKET / name, target)
|
||||
prior = Path('research/evaluations/language-pilot-01/scenarios.json')
|
||||
(tmp_path / prior).parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copy(ROOT / prior, tmp_path / prior)
|
||||
for relative in replication.DEPENDENCIES:
|
||||
(tmp_path / relative).parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copy(ROOT / relative, tmp_path / relative)
|
||||
replication.prepare(tmp_path)
|
||||
return tmp_path
|
||||
|
||||
|
||||
def test_preparation_checks_equivalence_isolation_and_counterbalance(packet):
|
||||
frozen, assignments, scenarios = replication.check(packet)
|
||||
assert len(assignments) == 12 and sum(len(s['criteria']) for s in scenarios) == 36
|
||||
guidance = replication.read(packet / replication.PACKET / 'guidance.json')
|
||||
for item in guidance['items']:
|
||||
for condition in ('patterns', 'prose'):
|
||||
assert (packet / replication.PACKET / f'pack-{condition}.md').read_text().count(item['body']) == 1
|
||||
for assignment in assignments:
|
||||
text = (packet / replication.PACKET / 'requests' / (assignment['run'] + '.txt')).read_text()
|
||||
scenario = next(s for s in scenarios if s['id'] == assignment['scenario'])
|
||||
assert text.endswith(scenario['brief'] + '\n')
|
||||
assert '"criteria"' not in text and '"prohibited"' not in text
|
||||
assert sum(text.count(s['brief']) for s in scenarios) == 1
|
||||
with pytest.raises(ValueError, match='already frozen'):
|
||||
replication.prepare(packet)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('path', ['requests/r01.txt', 'pack-prose.md', 'scores-template.json'])
|
||||
def test_tampered_prompt_guidance_or_blank_sheet_is_rejected(packet, path):
|
||||
p = packet / replication.PACKET / path
|
||||
p.write_text(p.read_text() + '\nchanged\n')
|
||||
with pytest.raises(ValueError, match='Frozen input changed'):
|
||||
replication.check(packet)
|
||||
|
||||
|
||||
def test_no_calls_before_frozen_inputs_are_committed(packet):
|
||||
with pytest.raises(subprocess.CalledProcessError):
|
||||
replication.run(12, packet, lambda: pytest.fail('No adapter before commit verification'))
|
||||
|
||||
|
||||
def test_call_ceiling_retains_failure_and_never_retries_reserved_sessions(packet, monkeypatch):
|
||||
monkeypatch.setattr(replication, 'committed', lambda *_: None)
|
||||
attempts = []
|
||||
class Adapter:
|
||||
endpoint = 'fixture:no-provider'
|
||||
config = SimpleNamespace(to_dict=lambda: {'max_tokens': 1400})
|
||||
responses = []
|
||||
rejected_content = None
|
||||
def generate(self, request):
|
||||
attempts.append(request.prompt)
|
||||
if len(attempts) == 2:
|
||||
raise RuntimeError('Retained synthetic failure')
|
||||
return SimpleNamespace(markdown='Synthetic fixture response; no empirical result.\n')
|
||||
with pytest.raises(ValueError, match='aggregate call ceiling'):
|
||||
replication.run(11, packet, Adapter)
|
||||
assert not attempts
|
||||
replication.run(12, packet, Adapter)
|
||||
assert len(attempts) == 12
|
||||
assert replication.read(packet / replication.PACKET / 'raw/r02/receipt.json')['status'] == 'failed'
|
||||
replication.run(12, packet, Adapter)
|
||||
assert len(attempts) == 12
|
||||
assert len(list((packet / replication.PACKET / 'masked').glob('*.md'))) == 11
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def scenarios():
|
||||
return replication.read(ROOT / replication.PACKET / 'scenarios.json')
|
||||
|
||||
|
||||
def reviewed_sheet(scenarios):
|
||||
sheet = replication.score_template(scenarios)
|
||||
sheet['reviewer'].update(name='Fixture reviewer', role='author-model', assignment_seen=False, prior_scores_seen=False)
|
||||
for answer in sheet['answers'].values():
|
||||
answer.update(available=True, unsupported_claims=[])
|
||||
for rating in answer['ratings'].values():
|
||||
rating.update(value=0, locator='Synthetic fixture answer', reason='Fixture miss, not an actual model judgment.')
|
||||
return sheet
|
||||
|
||||
|
||||
def test_blank_template_keeps_unreviewed_values_null(scenarios):
|
||||
sheet = replication.score_template(scenarios)
|
||||
replication.scores_check(sheet, scenarios)
|
||||
assert all(r['value'] is None for a in sheet['answers'].values() for r in a['ratings'].values())
|
||||
with pytest.raises(ValueError, match='Reviewer provenance'):
|
||||
replication.scores_check(sheet, scenarios, complete=True)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('mutation,match', [
|
||||
(lambda s: next(iter(s['answers'].values()))['ratings']['A1'].update(value=0.5), 'binary'),
|
||||
(lambda s: next(iter(s['answers'].values()))['ratings']['A1'].update(value=True), 'binary'),
|
||||
(lambda s: next(iter(s['answers'].values()))['ratings']['A1'].update(reason=None), 'evidence and reason'),
|
||||
(lambda s: next(iter(s['answers'].values())).update(available=False), 'cannot be scored zero'),
|
||||
(lambda s: next(iter(s['answers'].values())).update(unsupported_claims=None), 'Unsupported-claim review'),
|
||||
(lambda s: s['reviewer'].update(independent_human=True), 'Model review cannot'),
|
||||
(lambda s: s['reviewer'].update(assignment_seen=None), 'Exposure disclosure'),
|
||||
(lambda s: next(iter(s['answers'].values()))['ratings'].pop('A6'), 'criterion inventory'),
|
||||
])
|
||||
def test_invalid_or_missing_score_evidence_cannot_become_validation(scenarios, mutation, match):
|
||||
sheet = reviewed_sheet(scenarios)
|
||||
mutation(sheet)
|
||||
with pytest.raises(ValueError, match=match):
|
||||
replication.scores_check(sheet, scenarios, complete=True)
|
||||
|
||||
|
||||
def test_failed_pair_is_excluded_and_retained_answers_are_verified(packet, scenarios, monkeypatch):
|
||||
monkeypatch.setattr(replication, 'committed', lambda *_: None)
|
||||
calls = []
|
||||
class Adapter:
|
||||
endpoint = 'fixture:no-provider'
|
||||
config = SimpleNamespace(to_dict=lambda: {})
|
||||
responses = []
|
||||
rejected_content = None
|
||||
def generate(self, request):
|
||||
calls.append(request)
|
||||
if len(calls) == 2:
|
||||
raise RuntimeError('Fixture failure')
|
||||
return SimpleNamespace(markdown='Fixture output.\n')
|
||||
replication.run(12, packet, Adapter)
|
||||
sheet = reviewed_sheet(scenarios)
|
||||
assignment = replication.read(packet / replication.PACKET / 'assignment.json')[1]
|
||||
unavailable = sheet['answers'][assignment['blind_id']]
|
||||
unavailable['available'] = False
|
||||
for rating in unavailable['ratings'].values():
|
||||
rating['value'] = None
|
||||
summary = replication.summarize(sheet, packet)
|
||||
assert not summary['pairs'][0]['included']
|
||||
assert all(t['criteria'] == 30 and t['completed_pairs'] == 5 for t in summary['totals'].values())
|
||||
first = replication.read(packet / replication.PACKET / 'assignment.json')[0]['blind_id']
|
||||
sheet['answers'][first]['ratings']['A1']['quote'] = 'Invented quote absent from output.'
|
||||
with pytest.raises(ValueError, match='Quoted score evidence'):
|
||||
replication.summarize(sheet, packet)
|
||||
sheet['answers'][first]['ratings']['A1']['quote'] = None
|
||||
(packet / replication.PACKET / 'raw/r01/answer.md').write_text('Modified output.\n')
|
||||
with pytest.raises(ValueError, match='Retained answer changed'):
|
||||
replication.summarize(sheet, packet)
|
||||
|
|
@ -312,3 +312,15 @@ integration still block it. T03's stale pre-scheduling blocker is replaced by it
|
|||
remaining execution/publication acceptance, including ACTIVITY-WP-0041-T03.
|
||||
All three tasks remain wait and this workplan remains blocked. No evidence gate,
|
||||
policy, budget, authority or parked research workplan is changed.
|
||||
|
||||
## Gap-work consumer discovery — 2026-10-04
|
||||
|
||||
The user requested implementation of the assessed gaps. A bounded State Hub scan
|
||||
joined 49 todo and 2 progress tasks to active/ready workplans and matched frontend,
|
||||
interaction, editor/navigation/offline/component/designbook/keyboard/accessibility
|
||||
text outside FEP. No candidate met that index filter. [Discovery receipt](../docs/evidence/2026-10-04-consumer-discovery.json)
|
||||
records method and limits: this is not proof that no eligible consumer exists anywhere.
|
||||
No external task was reopened or invented, and no synthetic event was enrolled.
|
||||
T01/T02 retain actual source-verified task selection and common-template integration.
|
||||
WP-0009's authorized fresh replication advances independent-validation preparation;
|
||||
it does not replace the consumer decision/outcome acceptance in this plan.
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ type: workplan
|
|||
title: "Obtain independent validation for the M1 seed gate"
|
||||
domain: infotech
|
||||
repo: frontend-patterns
|
||||
status: backlog
|
||||
status: active
|
||||
owner: codex
|
||||
topic_slug: canon
|
||||
origin: residual
|
||||
|
|
@ -43,7 +43,7 @@ submitted-review-01 worksheet was a model review and does not count.
|
|||
|
||||
```task
|
||||
id: FEP-WP-0009-T02
|
||||
status: todo
|
||||
status: progress
|
||||
priority: medium
|
||||
state_hub_task_id: "8e0ac763-eba4-5625-ad63-af53322bcac4"
|
||||
```
|
||||
|
|
@ -76,3 +76,21 @@ at T01's human gate here, or at external owners (GLAS-WP-0012, ACTIVITY-WP-0041)
|
|||
an actual consumer for WP-0008. Those plans stay blocked; no task or workplan opened.
|
||||
|
||||
2026-10-04 CUST-WP-0075 owner routing: `FEP-WP-0009-T01` remains a human wait under open decision `6f698650-de45-4402-a84f-ea19ae5f96af`. Exact scoped human act and native receipts remain required; this metadata records no approval or execution.
|
||||
|
||||
## Gap implementation resumed — 2026-10-04
|
||||
|
||||
The user asked to work the assessed gaps, reopening this parked plan. Coverage audit:
|
||||
WP-0008 owns real consumers/outcomes and admitted improvement; WP-0009 owns independent
|
||||
validation/replication; WP-0003/0004 owns broader/gated applicability; WP-0006 owns
|
||||
comparable learning. No uncovered ownership requires a duplicate workplan.
|
||||
|
||||
T02 starts with a fresh replication packet: authored variants separate from pilot-01,
|
||||
atomic binary criteria, equivalent named/prose guidance, frozen release and input pins,
|
||||
condition-isolated prompts and evidence-bearing scoring validation. Preparation makes
|
||||
no inference call and cannot satisfy the independent human gate. Explicit paid-call
|
||||
budget is requested separately; absent authorization, execution remains pending.
|
||||
|
||||
The user then explicitly authorized **up to 12 calls, 1,400 output tokens per call,
|
||||
no retries**. This is the fresh replication budget only, not a grant for further model
|
||||
reviews, autonomous editing, policy changes or human-score substitution. Freeze and
|
||||
commit the packet/runner before execution; reserve every attempt and retain failures.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue