markitect-main/examples/infospace-with-history/docs/advanced-usage.md

# Advanced Usage — Wealth of Nations Infospace

Patterns for working with the WoN infospace (988 entities) after the initial
pipeline run. Every command in this file has been run against the actual
infospace at the time of writing (2026-04-21); output shapes are excerpted
verbatim.

All commands assume `cwd = examples/infospace-with-history` and the
`markitect-venv` Python environment.

---

## 1. Incremental evaluation — add entities after the initial run

`markitect infospace evaluate` writes one file per entity under
`output/evaluations/<slug>.md`. It skips any entity whose evaluation file
already exists, so re-running after adding a new entity processes only the
new one.

```bash
# Add a new entity file
vim output/entities/new-concept.md

# Evaluate only the new entity (explicit)
markitect infospace evaluate --entity new-concept --provider openrouter

# Or re-run the whole pass — existing 988 are skipped, only the new file hits the LLM
markitect infospace evaluate --provider openrouter
```

**How skip detection works.** Evaluation slugs are normalised to underscores
with `_s_` preserving apostrophes (`farmers-capital` entity →
`farmer_s_capital.md` evaluation). If a new entity slug collides with an
existing evaluation under this normalisation, the eval will be skipped.
To be sure an entity was picked up, check:

```bash
# Count entities vs evaluations
ls output/entities/*.md | grep -Ev 'book-[0-9]+-(chapter-[0-9]+|introduction)-' | wc -l
ls output/evaluations/*.md | wc -l
```

---

## 2. Re-evaluating after guideline changes

`evaluate` has no `--force` flag; re-evaluation requires deleting the
existing file first.

```bash
# Re-evaluate a single entity after updating the evaluation rubric
rm output/evaluations/accumulation_of_stock.md
markitect infospace evaluate --entity accumulation-of-stock --provider openrouter

# Re-evaluate a whole chapter
ls output/entities/book-1-chapter-06-entities.md   # see which entities the chapter produced
# Map chapter entities to eval filenames (apostrophe/underscore normalisation) and rm them
```

After re-evaluating, refresh the aggregate:

```bash
markitect infospace eval-summary --update-metrics
```

This merges `per_entity_mean` into `output/metrics/metrics.yaml` so the next
`markitect infospace viability` check reflects the new scores.

---

## 3. Interpreting per-entity score distributions

`eval-summary` shows the mean for each of the five evaluation dimensions
plus the overall range:

```
$ markitect infospace eval-summary
Evaluation summary — 985 entities evaluated

  Dimension                        Mean
  --------------------------------------
  overall                         3.956
  definition_precision            3.620
  domain_placement                4.559
  explanatory_value               3.936
  source_grounding                4.358
  vsm_relevance                   3.305

  Range: 1.00 – 4.80
```

Interpretation:
- `overall` above the 3.5 viability threshold → the collection passes
  `per_entity_mean`.
- The lowest dimension (`vsm_relevance` = 3.305) is the weakest signal. If
  the collection is meant to be VSM-grounded, this is the dimension most
  worth improving (via sharper entity definitions or schema changes).
- A wide range (1.00 – 4.80) tells you there are outliers at both ends —
  worth triaging (see pattern 4).

---

## 4. Triaging low scorers

`markitect infospace entities --by-type` prints each entity's star score
in-line:

```
$ markitect infospace entities --by-type | head
=== Element (315 entities) ===
  active_and_productive_stock              Accumulation       S1   ★4.6
  advanced_state_of_society                General Theory     S5
  agio_of_bank_money                       Exchange           S2   ★4.8
```

Entities with no `★` have no evaluation yet. To list the lowest-scoring
entities across the whole collection:

```bash
# Extract overall_score from every evaluation file and sort ascending
for f in output/evaluations/*.md; do
  score=$(awk '/^overall_score:/ {print $2; exit}' "$f")
  printf "%s\t%s\n" "$score" "$(basename "$f" .md)"
done | sort -n | head -20
```

The 20 lowest scorers are the natural triage list — inspect their
`output/entities/<slug>.md` and evaluation rationales to decide whether to
refine the entity, merge it with a better-formed neighbour, or drop it.

---

## 5. Reading and acting on collection-check output

`markitect infospace check` runs five concerns (C1–C5). Use `--concern` to
focus on one and `--json` for machine-readable output:

```bash
# Redundancy — which pairs of entities are suspiciously similar?
markitect infospace check --concern redundancy --json
```

```json
{
  "redundancy": {
    "concern": "C1",
    "redundancy_ratio": 0.0061,
    "similar_pairs": [
      {"entity_a": "bank_economic_contribution_metrics",
       "entity_b": "bank_economic_development_metrics",
       "similarity": 1.0, "method": "word_overlap"},
      {"entity_a": "economic_system_objectives",
       "entity_b": "economic_system_purpose",
       "similarity": 0.9394, "method": "word_overlap"}
    ]
  }
}
```

Acting on this:
- **Similarity = 1.0** is almost certainly a duplicate — pick one slug and
  merge or delete the other.
- **0.85–0.99** usually means two entities genuinely cover the same idea
  with slight phrasing differences. Merging is the cleanest fix.
- **< 0.85** usually represents legitimate adjacent concepts — leave as-is
  unless the definition rubric says otherwise.

For coverage and coherence, the pattern is the same: the `--json` output
surfaces the specific entities / missing links / disconnected components
you need to look at, rather than a bare ratio.

---

## See also

- `METRICS-METHODOLOGY.md` — how each metric is computed.
- `docs/composition-guide.md` — using this infospace as a discipline for a
  different domain.
- `docs/performance-notes.md` — observed timings and provider choices.