feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on first pass; 3 failed (network errors). eval-summary --update-metrics written with per_entity_mean=3.9556. Viability dashboard: 6/6 PASS redundancy_ratio 0.0061 (max 0.10) coverage_ratio 0.6190 (min 0.40) coherence_comps 0.0000 (max 3) consistency_cycles 0.0000 (max 0) granularity_entropy 2.6748 (min 1.0) per_entity_mean 3.9556 (min 3.5) Dimension breakdown (mean across 985 entities): definition_precision 3.62 source_grounding 4.36 domain_placement 4.56 vsm_relevance 3.31 explanatory_value 3.94 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
81a4c8796a
commit
a9ca0adfcf
986 changed files with 63216 additions and 1 deletions
|
|
@ -0,0 +1,63 @@
|
|||
---
|
||||
entity_slug: idle_consumers
|
||||
evaluator: null
|
||||
evaluated_at: '2026-02-23T05:35:44.978016'
|
||||
overall_score: 3.4
|
||||
scores:
|
||||
- name: definition_precision
|
||||
value: 3.0
|
||||
max_value: 5.0
|
||||
rationale: The definition clearly identifies a distinct group (non-productive consumers)
|
||||
but relies somewhat on the circular concept of "without contributing to production
|
||||
through labour." The distinction between productive and unproductive consumption
|
||||
could be more precisely delineated.
|
||||
- name: source_grounding
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: This concept is well-grounded in Smith's actual discussion of productive
|
||||
vs. unproductive labour and consumption patterns in Book I, Chapter 6. Smith does
|
||||
explicitly discuss how different classes consume the annual produce without contributing
|
||||
to its creation.
|
||||
- name: domain_placement
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: '"Consumption" is an appropriate domain placement since this entity specifically
|
||||
concerns consumption patterns and their economic effects. The focus on consumption
|
||||
behavior rather than production processes makes this categorization accurate.'
|
||||
- name: vsm_relevance
|
||||
value: 2.0
|
||||
max_value: 5.0
|
||||
rationale: This entity doesn't map naturally to any specific VSM system as it represents
|
||||
a passive economic actor rather than an active system function. It's more of a
|
||||
constraint or environmental factor that affects system performance rather than
|
||||
a viable system component.
|
||||
- name: explanatory_value
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: This entity provides significant explanatory power by illuminating the
|
||||
structural relationship between productive capacity and consumption patterns.
|
||||
It helps explain the mechanism by which economic growth can be constrained or
|
||||
reversed through unproductive consumption.
|
||||
---
|
||||
|
||||
# Evaluation: Idle Consumers
|
||||
|
||||
## definition_precision — 3.0 / 5.0
|
||||
|
||||
The definition clearly identifies a distinct group (non-productive consumers) but relies somewhat on the circular concept of "without contributing to production through labour." The distinction between productive and unproductive consumption could be more precisely delineated.
|
||||
|
||||
## source_grounding — 4.0 / 5.0
|
||||
|
||||
This concept is well-grounded in Smith's actual discussion of productive vs. unproductive labour and consumption patterns in Book I, Chapter 6. Smith does explicitly discuss how different classes consume the annual produce without contributing to its creation.
|
||||
|
||||
## domain_placement — 4.0 / 5.0
|
||||
|
||||
"Consumption" is an appropriate domain placement since this entity specifically concerns consumption patterns and their economic effects. The focus on consumption behavior rather than production processes makes this categorization accurate.
|
||||
|
||||
## vsm_relevance — 2.0 / 5.0
|
||||
|
||||
This entity doesn't map naturally to any specific VSM system as it represents a passive economic actor rather than an active system function. It's more of a constraint or environmental factor that affects system performance rather than a viable system component.
|
||||
|
||||
## explanatory_value — 4.0 / 5.0
|
||||
|
||||
This entity provides significant explanatory power by illuminating the structural relationship between productive capacity and consumption patterns. It helps explain the mechanism by which economic growth can be constrained or reversed through unproductive consumption.
|
||||
Loading…
Add table
Add a link
Reference in a new issue