feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on first pass; 3 failed (network errors). eval-summary --update-metrics written with per_entity_mean=3.9556. Viability dashboard: 6/6 PASS redundancy_ratio 0.0061 (max 0.10) coverage_ratio 0.6190 (min 0.40) coherence_comps 0.0000 (max 3) consistency_cycles 0.0000 (max 0) granularity_entropy 2.6748 (min 1.0) per_entity_mean 3.9556 (min 3.5) Dimension breakdown (mean across 985 entities): definition_precision 3.62 source_grounding 4.36 domain_placement 4.56 vsm_relevance 3.31 explanatory_value 3.94 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
81a4c8796a
commit
a9ca0adfcf
986 changed files with 63216 additions and 1 deletions
|
|
@ -0,0 +1,64 @@
|
|||
---
|
||||
entity_slug: economic_system_adaptability
|
||||
evaluator: null
|
||||
evaluated_at: '2026-02-23T05:12:23.553622'
|
||||
overall_score: 3.2
|
||||
scores:
|
||||
- name: definition_precision
|
||||
value: 3.0
|
||||
max_value: 5.0
|
||||
rationale: The definition captures a coherent concept about systems adjusting to
|
||||
change, but uses somewhat vague terms like "changing circumstances" and "remain
|
||||
effective." It's reasonably precise but could be more specific about what constitutes
|
||||
adaptability versus mere change.
|
||||
- name: source_grounding
|
||||
value: 2.0
|
||||
max_value: 5.0
|
||||
rationale: While Smith does discuss how different systems suit different nations
|
||||
and ages, the specific concept of "adaptability" as a systematic capacity is more
|
||||
of an interpretive extrapolation than something Smith explicitly theorizes. The
|
||||
source grounding is weak and relies heavily on inference.
|
||||
- name: domain_placement
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: '"General Theory" is appropriate since this concept would apply across
|
||||
Smith''s various economic discussions rather than being specific to trade, production,
|
||||
or other particular domains. The placement correctly recognizes its broad theoretical
|
||||
nature.'
|
||||
- name: vsm_relevance
|
||||
value: 5.0
|
||||
max_value: 5.0
|
||||
rationale: This entity maps very naturally to S4 (intelligence/environmental adaptation)
|
||||
in the VSM, which is precisely concerned with how systems adapt to environmental
|
||||
changes and maintain viability over time. It's a clear VSM fit rather than being
|
||||
too abstract.
|
||||
- name: explanatory_value
|
||||
value: 2.0
|
||||
max_value: 5.0
|
||||
rationale: While adaptability is important, this entity primarily names a desirable
|
||||
property rather than explaining the mechanisms by which economic systems actually
|
||||
adapt. It lacks specificity about the structural relations or processes that enable
|
||||
adaptation.
|
||||
---
|
||||
|
||||
# Evaluation: Economic System Adaptability
|
||||
|
||||
## definition_precision — 3.0 / 5.0
|
||||
|
||||
The definition captures a coherent concept about systems adjusting to change, but uses somewhat vague terms like "changing circumstances" and "remain effective." It's reasonably precise but could be more specific about what constitutes adaptability versus mere change.
|
||||
|
||||
## source_grounding — 2.0 / 5.0
|
||||
|
||||
While Smith does discuss how different systems suit different nations and ages, the specific concept of "adaptability" as a systematic capacity is more of an interpretive extrapolation than something Smith explicitly theorizes. The source grounding is weak and relies heavily on inference.
|
||||
|
||||
## domain_placement — 4.0 / 5.0
|
||||
|
||||
"General Theory" is appropriate since this concept would apply across Smith's various economic discussions rather than being specific to trade, production, or other particular domains. The placement correctly recognizes its broad theoretical nature.
|
||||
|
||||
## vsm_relevance — 5.0 / 5.0
|
||||
|
||||
This entity maps very naturally to S4 (intelligence/environmental adaptation) in the VSM, which is precisely concerned with how systems adapt to environmental changes and maintain viability over time. It's a clear VSM fit rather than being too abstract.
|
||||
|
||||
## explanatory_value — 2.0 / 5.0
|
||||
|
||||
While adaptability is important, this entity primarily names a desirable property rather than explaining the mechanisms by which economic systems actually adapt. It lacks specificity about the structural relations or processes that enable adaptation.
|
||||
Loading…
Add table
Add a link
Reference in a new issue