feat(presentation): project Lefevre into a content-explorer snapshot
Map the Lefevre infospace to a schema 0.1 snapshot and presentation manifest. Category kinds, Gutenberg rights, and chunk provenance stay on the projection. Relation blocks are included only when both ends match a stored entity title. Assistant: grok Assistant-Session: 01a137af-2bea-7940-9a65-acd892344a72
This commit is contained in:
parent
f001b0c1cf
commit
fd0dc5fc02
9 changed files with 1421 additions and 22 deletions
|
|
@ -2,20 +2,12 @@
|
|||
# Custodian Brief — infospace-bench
|
||||
|
||||
**Domain:** communication
|
||||
**Last synced:** 2026-10-14 00:07 UTC
|
||||
**Last synced:** 2026-10-14 00:28 UTC
|
||||
**State Hub:** http://127.0.0.1:8000 *(adjust if running on a remote machine)*
|
||||
|
||||
## Active Workstreams
|
||||
|
||||
### Bind infospaces to content-explorer as the presentation UI
|
||||
Progress: 0/5 done | workplan_id: `8b7261bb-9264-515c-948f-b6384e74fdfd`
|
||||
|
||||
**Open tasks:**
|
||||
- · T01 — Specify the reusable projection rule `128d6123`
|
||||
- · T02 — Project the Lefevre snapshot `5861057c`
|
||||
- · T03 — Write the Lefevre presentation manifest `922bddc1`
|
||||
- · T04 — Validate the Lefevre pair against the publication rules `f85eae30`
|
||||
- · T05 — Record the reusable rule beside the example `3852fcf3`
|
||||
*(none — repo may need first-session setup)*
|
||||
|
||||
## Inbox Hygiene
|
||||
|
||||
|
|
|
|||
|
|
@ -29,7 +29,7 @@
|
|||
| workplan | IB-WP-0019 | done | — | workplans/IB-WP-0019-budget-and-usage-registry.md |
|
||||
| workplan | IB-WP-0020 | finished | — | workplans/IB-WP-0020-provider-routing-cli.md |
|
||||
| workplan | IB-WP-0021 | finished | — | workplans/IB-WP-0021-scope-intent-reconciliation.md |
|
||||
| workplan | IB-WP-0022 | ready | — | workplans/IB-WP-0022-content-explorer-presentation-binding.md |
|
||||
| workplan | IB-WP-0022 | finished | — | workplans/IB-WP-0022-content-explorer-presentation-binding.md |
|
||||
| task | IB-WP-0001-T01 | done | — | workplans/IB-WP-0001-state-hub-and-repo-bootstrap.md |
|
||||
| task | IB-WP-0001-T02 | done | — | workplans/IB-WP-0001-state-hub-and-repo-bootstrap.md |
|
||||
| task | IB-WP-0001-T03 | done | — | workplans/IB-WP-0001-state-hub-and-repo-bootstrap.md |
|
||||
|
|
@ -129,8 +129,8 @@
|
|||
| task | IB-WP-0021-T04 | done | — | workplans/IB-WP-0021-scope-intent-reconciliation.md |
|
||||
| task | IB-WP-0021-T05 | done | — | workplans/IB-WP-0021-scope-intent-reconciliation.md |
|
||||
| task | IB-WP-0021-T06 | done | — | workplans/IB-WP-0021-scope-intent-reconciliation.md |
|
||||
| task | IB-WP-0022-T01 | todo | — | workplans/IB-WP-0022-content-explorer-presentation-binding.md |
|
||||
| task | IB-WP-0022-T02 | todo | — | workplans/IB-WP-0022-content-explorer-presentation-binding.md |
|
||||
| task | IB-WP-0022-T03 | todo | — | workplans/IB-WP-0022-content-explorer-presentation-binding.md |
|
||||
| task | IB-WP-0022-T04 | todo | — | workplans/IB-WP-0022-content-explorer-presentation-binding.md |
|
||||
| task | IB-WP-0022-T05 | todo | — | workplans/IB-WP-0022-content-explorer-presentation-binding.md |
|
||||
| task | IB-WP-0022-T01 | done | — | workplans/IB-WP-0022-content-explorer-presentation-binding.md |
|
||||
| task | IB-WP-0022-T02 | done | — | workplans/IB-WP-0022-content-explorer-presentation-binding.md |
|
||||
| task | IB-WP-0022-T03 | done | — | workplans/IB-WP-0022-content-explorer-presentation-binding.md |
|
||||
| task | IB-WP-0022-T04 | done | — | workplans/IB-WP-0022-content-explorer-presentation-binding.md |
|
||||
| task | IB-WP-0022-T05 | done | — | workplans/IB-WP-0022-content-explorer-presentation-binding.md |
|
||||
|
|
|
|||
|
|
@ -0,0 +1,49 @@
|
|||
# Content-explorer projection
|
||||
|
||||
`lefevre-reminiscences-of-a-stock-operator` is the worked example of the
|
||||
reusable infospace projection. A later infospace uses the same function,
|
||||
`infospace_bench.presentation.project_infospace`, and the same snapshot and
|
||||
manifest shape. It does not get a separate screen.
|
||||
|
||||
Infospace-bench remains the system of record. Content-explorer only presents
|
||||
artifacts infospace-bench still owns. The files in this directory are a
|
||||
projection, not a second copy of the knowledge.
|
||||
|
||||
## Rule
|
||||
|
||||
One infospace becomes one content-explorer snapshot (schema version 0.1) and
|
||||
one presentation manifest (schema version 0.1).
|
||||
|
||||
- The dataspace id is the infospace slug.
|
||||
- Each rights-bearing work becomes one `sources[]` row. `license` is the
|
||||
stored rights string, `uri` is `source_url` or the identifier, and `title`
|
||||
is the work title. Chunk digests and original paths stay in that source’s
|
||||
`attribution` provenance. They are not presentation kinds.
|
||||
- Each `kind: entity` file becomes one entity. `kindId` is the `## Category`
|
||||
value normalized with `slugify` (`evidence_bearing_claim` becomes
|
||||
`evidence-bearing-claim`, and `event` stays `event`). The id is
|
||||
`urn:ce:<slug>:<kindId>:<key>`. The title comes from the heading, without a
|
||||
leading `Entity Title:` label. `tldr` and `summary` come from
|
||||
`## Definition`. `sourceRefs` point at the work-level source. Provenance
|
||||
(`workflow_id`, `stage_id`, `input_artifact_id`, `provider`) is copied onto
|
||||
`properties`.
|
||||
- Storage kinds `source`, `entity`, `relation`, and `generated` are not
|
||||
presentation kinds. Generated summaries, entity bundles, and evaluations
|
||||
stay out of the browse set.
|
||||
- A relation block is included only when both subject and object match one
|
||||
stored entity title under `slugify` (the same normalizer already used for
|
||||
entity keys). That is exact normalized equality, not a fuzzy or partial
|
||||
match. The snapshot `predicate` is `## Relation Type` passed through
|
||||
`slugify` (`actor_venue` becomes `actor-venue`). The free-text
|
||||
`## Predicate` is not the predicate id. Unresolved or ambiguous ends are
|
||||
quarantined and omitted from the snapshot.
|
||||
- The manifest carries kind labels, `kindViews` columns and facets, featured
|
||||
kinds, navigation, `relationLabels` forward and inverse wording for every
|
||||
predicate the snapshot actually emits, panels limited to `summary`,
|
||||
`relations`, `evidence`, and `claims`, and a `factPolicy`. No
|
||||
content-explorer UI code is required.
|
||||
|
||||
`snapshot.json` and `manifest.json` beside this note are the Lefevre
|
||||
projection produced by that function. The published-example minimum (at least
|
||||
three kinds and eight entities) is met by this mapping. The raw `artifacts/`
|
||||
tree is not a snapshot.
|
||||
|
|
@ -0,0 +1,127 @@
|
|||
{
|
||||
"factPolicy": "show-all",
|
||||
"home": {
|
||||
"featuredKinds": [
|
||||
"error",
|
||||
"event",
|
||||
"evidence-bearing-claim",
|
||||
"institution",
|
||||
"instrument",
|
||||
"market",
|
||||
"strategy",
|
||||
"trader"
|
||||
],
|
||||
"heading": "Explore Reminiscences of a Stock Operator"
|
||||
},
|
||||
"id": "lefevre-reminiscences-of-a-stock-operator",
|
||||
"kindViews": {
|
||||
"error": {
|
||||
"columns": [
|
||||
"name",
|
||||
"tldr",
|
||||
"tags"
|
||||
],
|
||||
"facets": [
|
||||
"tags"
|
||||
]
|
||||
},
|
||||
"event": {
|
||||
"columns": [
|
||||
"name",
|
||||
"tldr",
|
||||
"tags"
|
||||
],
|
||||
"facets": [
|
||||
"tags"
|
||||
]
|
||||
},
|
||||
"evidence-bearing-claim": {
|
||||
"columns": [
|
||||
"name",
|
||||
"tldr",
|
||||
"tags"
|
||||
],
|
||||
"facets": [
|
||||
"tags"
|
||||
]
|
||||
},
|
||||
"institution": {
|
||||
"columns": [
|
||||
"name",
|
||||
"tldr",
|
||||
"tags"
|
||||
],
|
||||
"facets": [
|
||||
"tags"
|
||||
]
|
||||
},
|
||||
"instrument": {
|
||||
"columns": [
|
||||
"name",
|
||||
"tldr",
|
||||
"tags"
|
||||
],
|
||||
"facets": [
|
||||
"tags"
|
||||
]
|
||||
},
|
||||
"market": {
|
||||
"columns": [
|
||||
"name",
|
||||
"tldr",
|
||||
"tags"
|
||||
],
|
||||
"facets": [
|
||||
"tags"
|
||||
]
|
||||
},
|
||||
"strategy": {
|
||||
"columns": [
|
||||
"name",
|
||||
"tldr",
|
||||
"tags"
|
||||
],
|
||||
"facets": [
|
||||
"tags"
|
||||
]
|
||||
},
|
||||
"trader": {
|
||||
"columns": [
|
||||
"name",
|
||||
"tldr",
|
||||
"tags"
|
||||
],
|
||||
"facets": [
|
||||
"tags"
|
||||
]
|
||||
}
|
||||
},
|
||||
"navigation": [
|
||||
{
|
||||
"kinds": [
|
||||
"error",
|
||||
"event",
|
||||
"evidence-bearing-claim",
|
||||
"institution",
|
||||
"instrument",
|
||||
"market",
|
||||
"strategy",
|
||||
"trader"
|
||||
],
|
||||
"label": "Browse"
|
||||
}
|
||||
],
|
||||
"panels": [
|
||||
"summary",
|
||||
"relations",
|
||||
"evidence",
|
||||
"claims"
|
||||
],
|
||||
"relationLabels": {
|
||||
"actor-venue": {
|
||||
"forward": "Acts in",
|
||||
"inverse": "Venue for"
|
||||
}
|
||||
},
|
||||
"schemaVersion": "0.1"
|
||||
}
|
||||
|
|
@ -0,0 +1,382 @@
|
|||
{
|
||||
"claims": [],
|
||||
"dataspace": {
|
||||
"description": "Presentation of the Reminiscences of a Stock Operator infospace (lefevre-reminiscences-of-a-stock-operator). Infospace-bench owns the artifacts. Content-explorer only presents them.",
|
||||
"id": "lefevre-reminiscences-of-a-stock-operator",
|
||||
"title": "Reminiscences of a Stock Operator"
|
||||
},
|
||||
"entities": [
|
||||
{
|
||||
"id": "urn:ce:lefevre-reminiscences-of-a-stock-operator:error:dollar-manipulation",
|
||||
"kindId": "error",
|
||||
"properties": {
|
||||
"artifact_id": "entity/entity-title-dollar-manipulation.md",
|
||||
"artifact_path": "artifacts/entities/entity-title-dollar-manipulation.md",
|
||||
"category": "error",
|
||||
"input_artifact_id": "source/chapter-01-part-006.md",
|
||||
"provider": "openrouter",
|
||||
"stage_id": "split-entities",
|
||||
"workflow_id": "generic-source-entities"
|
||||
},
|
||||
"sourceRefs": [
|
||||
"src:lefevre-reminiscences-of-a-stock-operator:http-www-gutenberg-org-60979"
|
||||
],
|
||||
"summary": "The practice of manipulating stock prices to extract profits from the market, often at the expense of average investors.",
|
||||
"tags": [
|
||||
"error"
|
||||
],
|
||||
"title": "Dollar Manipulation",
|
||||
"tldr": "The practice of manipulating stock prices to extract profits from the market, often at the expense of average investors."
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:lefevre-reminiscences-of-a-stock-operator:event:bryan-panic-of-96",
|
||||
"kindId": "event",
|
||||
"properties": {
|
||||
"artifact_id": "entity/entity-title-bryan-panic-of-96.md",
|
||||
"artifact_path": "artifacts/entities/entity-title-bryan-panic-of-96.md",
|
||||
"category": "event",
|
||||
"input_artifact_id": "source/chapter-01-part-006.md",
|
||||
"provider": "openrouter",
|
||||
"stage_id": "split-entities",
|
||||
"workflow_id": "generic-source-entities"
|
||||
},
|
||||
"sourceRefs": [
|
||||
"src:lefevre-reminiscences-of-a-stock-operator:http-www-gutenberg-org-60979"
|
||||
],
|
||||
"summary": "The Bryan Panic of 1896 was a financial crisis characterized by significant volatility and manipulation in the markets, leading to widespread impact on traders, including those in bucket shops.",
|
||||
"tags": [
|
||||
"event"
|
||||
],
|
||||
"title": "Bryan Panic of ’96",
|
||||
"tldr": "The Bryan Panic of 1896 was a financial crisis characterized by significant volatility and manipulation in the markets, leading to widespread impact on traders, including those in bucket shops."
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:lefevre-reminiscences-of-a-stock-operator:evidence-bearing-claim:bull-tips",
|
||||
"kindId": "evidence-bearing-claim",
|
||||
"properties": {
|
||||
"artifact_id": "entity/entity-title-bull-tips.md",
|
||||
"artifact_path": "artifacts/entities/entity-title-bull-tips.md",
|
||||
"category": "evidence_bearing_claim",
|
||||
"input_artifact_id": "source/chapter-01-part-006.md",
|
||||
"provider": "openrouter",
|
||||
"stage_id": "split-entities",
|
||||
"workflow_id": "generic-source-entities"
|
||||
},
|
||||
"sourceRefs": [
|
||||
"src:lefevre-reminiscences-of-a-stock-operator:http-www-gutenberg-org-60979"
|
||||
],
|
||||
"summary": "Bull tips are recommendations or information suggesting that a stock's price will rise, encouraging traders to buy. These were used strategically to influence market movements in the discussed scenario.",
|
||||
"tags": [
|
||||
"evidence-bearing-claim"
|
||||
],
|
||||
"title": "Bull Tips",
|
||||
"tldr": "Bull tips are recommendations or information suggesting that a stock's price will rise, encouraging traders to buy."
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:lefevre-reminiscences-of-a-stock-operator:institution:cosmopolitan",
|
||||
"kindId": "institution",
|
||||
"properties": {
|
||||
"artifact_id": "entity/entity-title-cosmopolitan.md",
|
||||
"artifact_path": "artifacts/entities/entity-title-cosmopolitan.md",
|
||||
"category": "institution",
|
||||
"input_artifact_id": "source/chapter-01-part-006.md",
|
||||
"provider": "openrouter",
|
||||
"stage_id": "split-entities",
|
||||
"workflow_id": "generic-source-entities"
|
||||
},
|
||||
"sourceRefs": [
|
||||
"src:lefevre-reminiscences-of-a-stock-operator:http-www-gutenberg-org-60979"
|
||||
],
|
||||
"summary": "The Cosmopolitan is identified as a bucket shop that engaged in manipulative practices, specifically targeting short positions in Sugar by artificially inflating stock prices.",
|
||||
"tags": [
|
||||
"institution"
|
||||
],
|
||||
"title": "Cosmopolitan",
|
||||
"tldr": "The Cosmopolitan is identified as a bucket shop that engaged in manipulative practices, specifically targeting short positions in Sugar by artificially inflating stock prices."
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:lefevre-reminiscences-of-a-stock-operator:instrument:sugar",
|
||||
"kindId": "instrument",
|
||||
"properties": {
|
||||
"artifact_id": "entity/entity-title-sugar.md",
|
||||
"artifact_path": "artifacts/entities/entity-title-sugar.md",
|
||||
"category": "instrument",
|
||||
"input_artifact_id": "source/chapter-01-part-006.md",
|
||||
"provider": "openrouter",
|
||||
"stage_id": "split-entities",
|
||||
"workflow_id": "generic-source-entities"
|
||||
},
|
||||
"sourceRefs": [
|
||||
"src:lefevre-reminiscences-of-a-stock-operator:http-www-gutenberg-org-60979"
|
||||
],
|
||||
"summary": "A specific stock that traders engage with, which is central to the narratives involving trading strategies and market manipulation.",
|
||||
"tags": [
|
||||
"instrument"
|
||||
],
|
||||
"title": "Sugar",
|
||||
"tldr": "A specific stock that traders engage with, which is central to the narratives involving trading strategies and market manipulation."
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:lefevre-reminiscences-of-a-stock-operator:instrument:sugar-commodity",
|
||||
"kindId": "instrument",
|
||||
"properties": {
|
||||
"artifact_id": "entity/entity-title-sugar-commodity.md",
|
||||
"artifact_path": "artifacts/entities/entity-title-sugar-commodity.md",
|
||||
"category": "instrument",
|
||||
"input_artifact_id": "source/chapter-01-part-006.md",
|
||||
"provider": "openrouter",
|
||||
"stage_id": "split-entities",
|
||||
"workflow_id": "generic-source-entities"
|
||||
},
|
||||
"sourceRefs": [
|
||||
"src:lefevre-reminiscences-of-a-stock-operator:http-www-gutenberg-org-60979"
|
||||
],
|
||||
"summary": "Sugar refers to a specific commodity that is traded on the market, in this case, associated with the incident involving Tom and Henry Williams, where their positions were affected by market manipulation.",
|
||||
"tags": [
|
||||
"instrument"
|
||||
],
|
||||
"title": "Sugar (Commodity)",
|
||||
"tldr": "Sugar refers to a specific commodity that is traded on the market, in this case, associated with the incident involving Tom and Henry Williams, where their positions were affected by market manipulation."
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:lefevre-reminiscences-of-a-stock-operator:instrument:western-union",
|
||||
"kindId": "instrument",
|
||||
"properties": {
|
||||
"artifact_id": "entity/entity-title-western-union.md",
|
||||
"artifact_path": "artifacts/entities/entity-title-western-union.md",
|
||||
"category": "instrument",
|
||||
"input_artifact_id": "source/chapter-01-part-006.md",
|
||||
"provider": "openrouter",
|
||||
"stage_id": "split-entities",
|
||||
"workflow_id": "generic-source-entities"
|
||||
},
|
||||
"sourceRefs": [
|
||||
"src:lefevre-reminiscences-of-a-stock-operator:http-www-gutenberg-org-60979"
|
||||
],
|
||||
"summary": "A specific stock favored for trading due to its liquidity and ease of price manipulation.",
|
||||
"tags": [
|
||||
"instrument"
|
||||
],
|
||||
"title": "Western Union",
|
||||
"tldr": "A specific stock favored for trading due to its liquidity and ease of price manipulation."
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:lefevre-reminiscences-of-a-stock-operator:instrument:western-union-stock",
|
||||
"kindId": "instrument",
|
||||
"properties": {
|
||||
"artifact_id": "entity/entity-title-western-union-stock.md",
|
||||
"artifact_path": "artifacts/entities/entity-title-western-union-stock.md",
|
||||
"category": "instrument",
|
||||
"input_artifact_id": "source/chapter-01-part-006.md",
|
||||
"provider": "openrouter",
|
||||
"stage_id": "split-entities",
|
||||
"workflow_id": "generic-source-entities"
|
||||
},
|
||||
"sourceRefs": [
|
||||
"src:lefevre-reminiscences-of-a-stock-operator:http-www-gutenberg-org-60979"
|
||||
],
|
||||
"summary": "Western Union is a stock that was frequently manipulated by traders due to its relative ease of movement within the market, making it a favored target for speculative trading schemes.",
|
||||
"tags": [
|
||||
"instrument"
|
||||
],
|
||||
"title": "Western Union (Stock)",
|
||||
"tldr": "Western Union is a stock that was frequently manipulated by traders due to its relative ease of movement within the market, making it a favored target for speculative trading schemes."
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:lefevre-reminiscences-of-a-stock-operator:market:bucket-shop",
|
||||
"kindId": "market",
|
||||
"properties": {
|
||||
"artifact_id": "entity/entity-title-bucket-shop.md",
|
||||
"artifact_path": "artifacts/entities/entity-title-bucket-shop.md",
|
||||
"category": "market",
|
||||
"input_artifact_id": "source/chapter-01-part-006.md",
|
||||
"provider": "openrouter",
|
||||
"stage_id": "split-entities",
|
||||
"workflow_id": "generic-source-entities"
|
||||
},
|
||||
"sourceRefs": [
|
||||
"src:lefevre-reminiscences-of-a-stock-operator:http-www-gutenberg-org-60979"
|
||||
],
|
||||
"summary": "A bucket shop is a type of brokerage that allows speculative trading in stocks without actually buying shares. These shops often manipulate market prices to exploit their clients, leading to significant losses for traders.",
|
||||
"tags": [
|
||||
"market"
|
||||
],
|
||||
"title": "Bucket Shop",
|
||||
"tldr": "A bucket shop is a type of brokerage that allows speculative trading in stocks without actually buying shares."
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:lefevre-reminiscences-of-a-stock-operator:market:new-york-stock-exchange",
|
||||
"kindId": "market",
|
||||
"properties": {
|
||||
"artifact_id": "entity/entity-title-new-york-stock-exchange.md",
|
||||
"artifact_path": "artifacts/entities/entity-title-new-york-stock-exchange.md",
|
||||
"category": "market",
|
||||
"input_artifact_id": "source/chapter-01-part-006.md",
|
||||
"provider": "openrouter",
|
||||
"stage_id": "split-entities",
|
||||
"workflow_id": "generic-source-entities"
|
||||
},
|
||||
"sourceRefs": [
|
||||
"src:lefevre-reminiscences-of-a-stock-operator:http-www-gutenberg-org-60979"
|
||||
],
|
||||
"summary": "The New York Stock Exchange (NYSE) is a major stock exchange where trading of stocks occurs, serving as a venue for some of the trading activities mentioned in the context of market manipulation.",
|
||||
"tags": [
|
||||
"market"
|
||||
],
|
||||
"title": "New York Stock Exchange",
|
||||
"tldr": "The New York Stock Exchange (NYSE) is a major stock exchange where trading of stocks occurs, serving as a venue for some of the trading activities mentioned in the context of market manipulation."
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:lefevre-reminiscences-of-a-stock-operator:strategy:bucket-shop-drive",
|
||||
"kindId": "strategy",
|
||||
"properties": {
|
||||
"artifact_id": "entity/entity-title-bucket-shop-drive.md",
|
||||
"artifact_path": "artifacts/entities/entity-title-bucket-shop-drive.md",
|
||||
"category": "strategy",
|
||||
"input_artifact_id": "source/chapter-01-part-006.md",
|
||||
"provider": "openrouter",
|
||||
"stage_id": "split-entities",
|
||||
"workflow_id": "generic-source-entities"
|
||||
},
|
||||
"sourceRefs": [
|
||||
"src:lefevre-reminiscences-of-a-stock-operator:http-www-gutenberg-org-60979"
|
||||
],
|
||||
"summary": "A bucket-shop drive refers to a situation where the prices of stocks are manipulated downwards in a coordinated effort to force clients who are long on those stocks to incur losses or be wiped out.",
|
||||
"tags": [
|
||||
"strategy"
|
||||
],
|
||||
"title": "Bucket-Shop Drive",
|
||||
"tldr": "A bucket-shop drive refers to a situation where the prices of stocks are manipulated downwards in a coordinated effort to force clients who are long on those stocks to incur losses or be wiped out."
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:lefevre-reminiscences-of-a-stock-operator:strategy:market-drive",
|
||||
"kindId": "strategy",
|
||||
"properties": {
|
||||
"artifact_id": "entity/entity-title-market-drive.md",
|
||||
"artifact_path": "artifacts/entities/entity-title-market-drive.md",
|
||||
"category": "strategy",
|
||||
"input_artifact_id": "source/chapter-01-part-006.md",
|
||||
"provider": "openrouter",
|
||||
"stage_id": "split-entities",
|
||||
"workflow_id": "generic-source-entities"
|
||||
},
|
||||
"sourceRefs": [
|
||||
"src:lefevre-reminiscences-of-a-stock-operator:http-www-gutenberg-org-60979"
|
||||
],
|
||||
"summary": "A tactic employed to artificially inflate or deflate stock prices in a way that leads to trader losses and profits for others.",
|
||||
"tags": [
|
||||
"strategy"
|
||||
],
|
||||
"title": "Market Drive",
|
||||
"tldr": "A tactic employed to artificially inflate or deflate stock prices in a way that leads to trader losses and profits for others."
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:lefevre-reminiscences-of-a-stock-operator:strategy:short-position",
|
||||
"kindId": "strategy",
|
||||
"properties": {
|
||||
"artifact_id": "entity/entity-title-short-position.md",
|
||||
"artifact_path": "artifacts/entities/entity-title-short-position.md",
|
||||
"category": "strategy",
|
||||
"input_artifact_id": "source/chapter-01-part-006.md",
|
||||
"provider": "openrouter",
|
||||
"stage_id": "split-entities",
|
||||
"workflow_id": "generic-source-entities"
|
||||
},
|
||||
"sourceRefs": [
|
||||
"src:lefevre-reminiscences-of-a-stock-operator:http-www-gutenberg-org-60979"
|
||||
],
|
||||
"summary": "A short position refers to the practice of selling stocks that a trader does not own, with the intention of buying them back at a lower price to profit from the decline. It was a significant factor in the losses suffered by the traders in the context.",
|
||||
"tags": [
|
||||
"strategy"
|
||||
],
|
||||
"title": "Short Position",
|
||||
"tldr": "A short position refers to the practice of selling stocks that a trader does not own, with the intention of buying them back at a lower price to profit from the decline."
|
||||
},
|
||||
{
|
||||
"id": "urn:ce:lefevre-reminiscences-of-a-stock-operator:trader:market-factor",
|
||||
"kindId": "trader",
|
||||
"properties": {
|
||||
"artifact_id": "entity/entity-title-market-factor.md",
|
||||
"artifact_path": "artifacts/entities/entity-title-market-factor.md",
|
||||
"category": "trader",
|
||||
"input_artifact_id": "source/chapter-01-part-006.md",
|
||||
"provider": "openrouter",
|
||||
"stage_id": "split-entities",
|
||||
"workflow_id": "generic-source-entities"
|
||||
},
|
||||
"sourceRefs": [
|
||||
"src:lefevre-reminiscences-of-a-stock-operator:http-www-gutenberg-org-60979"
|
||||
],
|
||||
"summary": "A market factor is an operator or trader who significantly influences trading conditions or stock prices within the market. In the text, a certain trader is identified as a market factor due to his previous successes in market manipulation.",
|
||||
"tags": [
|
||||
"trader"
|
||||
],
|
||||
"title": "Market Factor",
|
||||
"tldr": "A market factor is an operator or trader who significantly influences trading conditions or stock prices within the market."
|
||||
}
|
||||
],
|
||||
"kinds": [
|
||||
{
|
||||
"id": "error",
|
||||
"label": "Error",
|
||||
"plural": "Errors"
|
||||
},
|
||||
{
|
||||
"id": "event",
|
||||
"label": "Event",
|
||||
"plural": "Events"
|
||||
},
|
||||
{
|
||||
"id": "evidence-bearing-claim",
|
||||
"label": "Evidence Bearing Claim",
|
||||
"plural": "Evidence Bearing Claims"
|
||||
},
|
||||
{
|
||||
"id": "institution",
|
||||
"label": "Institution",
|
||||
"plural": "Institutions"
|
||||
},
|
||||
{
|
||||
"id": "instrument",
|
||||
"label": "Instrument",
|
||||
"plural": "Instruments"
|
||||
},
|
||||
{
|
||||
"id": "market",
|
||||
"label": "Market",
|
||||
"plural": "Markets"
|
||||
},
|
||||
{
|
||||
"id": "strategy",
|
||||
"label": "Strategy",
|
||||
"plural": "Strategys"
|
||||
},
|
||||
{
|
||||
"id": "trader",
|
||||
"label": "Trader",
|
||||
"plural": "Traders"
|
||||
}
|
||||
],
|
||||
"relations": [
|
||||
{
|
||||
"from": "urn:ce:lefevre-reminiscences-of-a-stock-operator:institution:cosmopolitan",
|
||||
"id": "rel:lefevre-reminiscences-of-a-stock-operator:0001",
|
||||
"predicate": "actor-venue",
|
||||
"sourceRefs": [
|
||||
"src:lefevre-reminiscences-of-a-stock-operator:http-www-gutenberg-org-60979"
|
||||
],
|
||||
"to": "urn:ce:lefevre-reminiscences-of-a-stock-operator:market:bucket-shop"
|
||||
}
|
||||
],
|
||||
"schemaVersion": "0.1",
|
||||
"sources": [
|
||||
{
|
||||
"attribution": "creator: Edwin Lefevre\nidentifier: http://www.gutenberg.org/60979\nprovenance:\n- chunk_id: chapter-01-part-001\n digest: 2692a29f1fccaa88f06bc6276a83f4ef81acca0b33dfec69dd68322e4789c19d\n original_path: /mnt/c/Users/bernd.worsch/Downloads/LefevreEdwin-ReminiscencesOfAStockOperator.epub!OEBPS/7612516312020751654_60979-h-2.htm.xhtml\n path: artifacts/sources/chapter-01-part-001.md\n- chunk_id: chapter-01-part-002\n digest: e8a019fbe892410af26b4aa83e50f437af0d7fd331a6bbf7967fa2bf7912994e\n original_path: /mnt/c/Users/bernd.worsch/Downloads/LefevreEdwin-ReminiscencesOfAStockOperator.epub!OEBPS/7612516312020751654_60979-h-2.htm.xhtml\n path: artifacts/sources/chapter-01-part-002.md\n- chunk_id: chapter-01-part-003\n digest: c6aa128750437df02ec56aa680dca328a365a6092543957993c19c15a0828af5\n original_path: /mnt/c/Users/bernd.worsch/Downloads/LefevreEdwin-ReminiscencesOfAStockOperator.epub!OEBPS/7612516312020751654_60979-h-2.htm.xhtml\n path: artifacts/sources/chapter-01-part-003.md\n- chunk_id: chapter-01-part-004\n digest: 1677ba1989d7940fea577bbaf450e5df633953e96d84e2f5bfcb93294b3991e8\n original_path: /mnt/c/Users/bernd.worsch/Downloads/LefevreEdwin-ReminiscencesOfAStockOperator.epub!OEBPS/7612516312020751654_60979-h-2.htm.xhtml\n path: artifacts/sources/chapter-01-part-004.md\n- chunk_id: chapter-01-part-005\n digest: b3a7c71f2e57f5ef5a5736c7b37ed93b9ae55507da6d04f7f0995700f1778880\n original_path: /mnt/c/Users/bernd.worsch/Downloads/LefevreEdwin-ReminiscencesOfAStockOperator.epub!OEBPS/7612516312020751654_60979-h-2.htm.xhtml\n path: artifacts/sources/chapter-01-part-005.md\n- chunk_id: chapter-01-part-006\n digest: a7e5c1bdb722d634cd874f1b03b7030d09640104aead008137ea0413ad8cee42\n original_path: /mnt/c/Users/bernd.worsch/Downloads/LefevreEdwin-ReminiscencesOfAStockOperator.epub!OEBPS/7612516312020751654_60979-h-2.htm.xhtml\n path: artifacts/sources/chapter-01-part-006.md",
|
||||
"id": "src:lefevre-reminiscences-of-a-stock-operator:http-www-gutenberg-org-60979",
|
||||
"license": "Public domain in the USA.",
|
||||
"retrievedAt": "2026-05-18",
|
||||
"title": "Reminiscences of a Stock Operator",
|
||||
"uri": "https://www.gutenberg.org/files/60979/60979-h/60979-h.htm"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -33,6 +33,7 @@ from .history import (
|
|||
from .inspection import export_mermaid, relationship_summary
|
||||
from .lifecycle import add_artifact, create_infospace, load_infospace
|
||||
from .markdown_adapter import validate_infospace_artifacts
|
||||
from .presentation import project_infospace, snapshot_digest, write_presentation
|
||||
from .semantics import list_entities, list_relations
|
||||
from .workflow import (
|
||||
FixtureAssistedGenerationAdapter,
|
||||
|
|
@ -382,6 +383,17 @@ def build_parser() -> argparse.ArgumentParser:
|
|||
engine_sync.add_argument("root")
|
||||
engine_sync.add_argument("--apply", action="store_true")
|
||||
|
||||
present = sub.add_parser(
|
||||
"present",
|
||||
help="Project an infospace to a content-explorer snapshot and manifest",
|
||||
)
|
||||
present.add_argument("root")
|
||||
present.add_argument(
|
||||
"--out",
|
||||
default="",
|
||||
help="Directory for snapshot.json and manifest.json. Omit to print only.",
|
||||
)
|
||||
|
||||
return parser
|
||||
|
||||
|
||||
|
|
@ -672,6 +684,21 @@ def main(argv: list[str] | None = None) -> int:
|
|||
force=args.force,
|
||||
)
|
||||
_write_json(result.to_dict())
|
||||
elif args.command == "present":
|
||||
if args.out:
|
||||
projection = write_presentation(Path(args.root), Path(args.out))
|
||||
else:
|
||||
projection = project_infospace(Path(args.root))
|
||||
_write_json(
|
||||
{
|
||||
"dataspace": projection.snapshot["dataspace"]["id"],
|
||||
"snapshot_sha256": snapshot_digest(projection.snapshot),
|
||||
"entity_ids": [item["id"] for item in projection.snapshot["entities"]],
|
||||
"relation_ids": [item["id"] for item in projection.snapshot["relations"]],
|
||||
"quarantined": len(projection.quarantined_relations),
|
||||
"out": args.out,
|
||||
}
|
||||
)
|
||||
elif args.command == "engine":
|
||||
if args.engine_command == "inspect":
|
||||
_write_json(
|
||||
|
|
|
|||
644
src/infospace_bench/presentation.py
Normal file
644
src/infospace_bench/presentation.py
Normal file
|
|
@ -0,0 +1,644 @@
|
|||
"""Project an infospace into a content-explorer snapshot and manifest.
|
||||
|
||||
Infospace-bench remains the system of record. The snapshot and manifest are
|
||||
a presentation of artifacts the infospace already stores. Content-explorer
|
||||
does not become a second knowledge store, and this module does not rewrite
|
||||
infospace files.
|
||||
|
||||
The worked example is ``lefevre-reminiscences-of-a-stock-operator``. A later
|
||||
infospace uses the same function: one work-level source per rights-bearing
|
||||
work, one entity per ``kind: entity`` file, and a relation only when both
|
||||
ends match a stored entity title exactly.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from markitect_tool import parse_markdown
|
||||
|
||||
from .errors import InfospaceError
|
||||
from .lifecycle import load_infospace
|
||||
from .markdown_adapter import ParsedMarkdownArtifact, extract_section_text
|
||||
from .models import KnowledgeArtifact
|
||||
from .semantics import slugify
|
||||
|
||||
SCHEMA_VERSION = "0.1"
|
||||
STORAGE_KINDS = frozenset({"source", "entity", "relation", "generated"})
|
||||
PANELS = ["summary", "relations", "evidence", "claims"]
|
||||
TITLE_LABEL = "entity title:"
|
||||
SLUG_RE = re.compile(r"^[a-z0-9-]+$")
|
||||
RULE_SPLIT = re.compile(r"(?m)^---\s*$")
|
||||
RELATION_LABELS = {
|
||||
"actor-venue": ("Acts in", "Venue for"),
|
||||
"cause-effect": ("Causes", "Caused by"),
|
||||
"strategy-outcome": ("Leads to", "Outcome of"),
|
||||
"risk-mitigation": ("Mitigates", "Mitigated by"),
|
||||
"lesson-evidence": ("Shown by", "Evidence of"),
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ResolvedRelation:
|
||||
artifact_id: str
|
||||
block_index: int
|
||||
subject: str
|
||||
object: str
|
||||
relation_type: str
|
||||
predicate_text: str
|
||||
predicate: str
|
||||
from_id: str
|
||||
to_id: str
|
||||
relation_id: str
|
||||
source_id: str
|
||||
|
||||
def to_dict(self) -> dict[str, str | int]:
|
||||
return {
|
||||
"artifact_id": self.artifact_id,
|
||||
"block_index": self.block_index,
|
||||
"subject": self.subject,
|
||||
"object": self.object,
|
||||
"relation_type": self.relation_type,
|
||||
"predicate_text": self.predicate_text,
|
||||
"predicate": self.predicate,
|
||||
"from_id": self.from_id,
|
||||
"to_id": self.to_id,
|
||||
"relation_id": self.relation_id,
|
||||
"source_id": self.source_id,
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class QuarantinedRelation:
|
||||
artifact_id: str
|
||||
block_index: int
|
||||
subject: str
|
||||
object: str
|
||||
relation_type: str
|
||||
predicate_text: str
|
||||
reason: str
|
||||
|
||||
def to_dict(self) -> dict[str, str | int]:
|
||||
return {
|
||||
"artifact_id": self.artifact_id,
|
||||
"block_index": self.block_index,
|
||||
"subject": self.subject,
|
||||
"object": self.object,
|
||||
"relation_type": self.relation_type,
|
||||
"predicate_text": self.predicate_text,
|
||||
"reason": self.reason,
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Projection:
|
||||
snapshot: dict[str, Any]
|
||||
manifest: dict[str, Any]
|
||||
resolved_relations: list[ResolvedRelation] = field(default_factory=list)
|
||||
quarantined_relations: list[QuarantinedRelation] = field(default_factory=list)
|
||||
|
||||
def to_dict(self) -> dict[str, Any]:
|
||||
return {
|
||||
"snapshot": self.snapshot,
|
||||
"manifest": self.manifest,
|
||||
"resolved_relations": [item.to_dict() for item in self.resolved_relations],
|
||||
"quarantined_relations": [
|
||||
item.to_dict() for item in self.quarantined_relations
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def project_infospace(root: str | Path) -> Projection:
|
||||
"""Map one infospace directory to a snapshot, a manifest, and quarantines."""
|
||||
|
||||
infospace = load_infospace(root)
|
||||
slug = infospace.config.slug
|
||||
if not SLUG_RE.fullmatch(slug):
|
||||
raise InfospaceError(
|
||||
"invalid_dataspace_id",
|
||||
f"Infospace slug is not a content-explorer dataspace id: {slug}",
|
||||
{"slug": slug},
|
||||
)
|
||||
|
||||
sources = [item for item in infospace.artifacts if item.kind == "source"]
|
||||
entities = [item for item in infospace.artifacts if item.kind == "entity"]
|
||||
relations = [item for item in infospace.artifacts if item.kind == "relation"]
|
||||
works, chunk_to_work = _work_sources(slug, sources)
|
||||
if not works:
|
||||
raise InfospaceError(
|
||||
"missing_work_source",
|
||||
"No rights-bearing source record to project",
|
||||
{"root": str(infospace.root)},
|
||||
)
|
||||
|
||||
projected_entities, kind_ids = _project_entities(
|
||||
infospace.root,
|
||||
slug,
|
||||
entities,
|
||||
chunk_to_work,
|
||||
works,
|
||||
)
|
||||
title_index = _title_index(projected_entities)
|
||||
resolved, quarantined = _project_relations(
|
||||
infospace.root,
|
||||
slug,
|
||||
relations,
|
||||
title_index,
|
||||
chunk_to_work,
|
||||
works,
|
||||
)
|
||||
snapshot = _snapshot(
|
||||
slug=slug,
|
||||
name=infospace.config.name,
|
||||
works=works,
|
||||
entities=projected_entities,
|
||||
resolved=resolved,
|
||||
kind_ids=kind_ids,
|
||||
)
|
||||
manifest = _manifest(slug, infospace.config.name, snapshot)
|
||||
return Projection(
|
||||
snapshot=snapshot,
|
||||
manifest=manifest,
|
||||
resolved_relations=resolved,
|
||||
quarantined_relations=quarantined,
|
||||
)
|
||||
|
||||
|
||||
def snapshot_digest(snapshot: dict[str, Any]) -> str:
|
||||
payload = json.dumps(
|
||||
snapshot,
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
ensure_ascii=False,
|
||||
)
|
||||
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def write_presentation(root: str | Path, destination: str | Path) -> Projection:
|
||||
"""Write ``snapshot.json`` and ``manifest.json`` without touching artifacts."""
|
||||
|
||||
projection = project_infospace(root)
|
||||
target = Path(destination)
|
||||
target.mkdir(parents=True, exist_ok=True)
|
||||
_write_json(target / "snapshot.json", projection.snapshot)
|
||||
_write_json(target / "manifest.json", projection.manifest)
|
||||
return projection
|
||||
|
||||
|
||||
def display_title(heading: str) -> str:
|
||||
"""Title stored in the heading. A leading ``Entity Title:`` label is not the name."""
|
||||
|
||||
text = " ".join(heading.strip().split())
|
||||
if text.lower().startswith(TITLE_LABEL):
|
||||
return text.split(":", 1)[1].strip()
|
||||
return text
|
||||
|
||||
|
||||
def _work_sources(
|
||||
slug: str,
|
||||
sources: list[KnowledgeArtifact],
|
||||
) -> tuple[list[dict[str, Any]], dict[str, str]]:
|
||||
grouped: dict[tuple[str, str, str], dict[str, Any]] = {}
|
||||
chunk_to_work: dict[str, str] = {}
|
||||
for artifact in sorted(sources, key=lambda item: item.id):
|
||||
book = dict(artifact.provenance.get("book_metadata") or {})
|
||||
rights = str(book.get("rights") or "").strip()
|
||||
uri = str(book.get("source_url") or book.get("identifier") or "").strip()
|
||||
title = str(book.get("title") or artifact.title or "").strip()
|
||||
if not rights or not uri or not title:
|
||||
raise InfospaceError(
|
||||
"incomplete_work_source",
|
||||
f"Source {artifact.id} is missing rights, uri, or title",
|
||||
{"artifact_id": artifact.id},
|
||||
)
|
||||
key = (uri, title, rights)
|
||||
work = grouped.get(key)
|
||||
if work is None:
|
||||
identifier = str(book.get("identifier") or uri)
|
||||
work_key = slugify(identifier) or "work"
|
||||
work = {
|
||||
"id": f"src:{slug}:{work_key}",
|
||||
"uri": uri,
|
||||
"title": title,
|
||||
"license": rights,
|
||||
"creator": str(book.get("creator") or "").strip(),
|
||||
"identifier": identifier,
|
||||
"retrieved_at": _date(str(artifact.provenance.get("imported_at") or "")),
|
||||
"chunks": [],
|
||||
}
|
||||
grouped[key] = work
|
||||
imported = _date(str(artifact.provenance.get("imported_at") or ""))
|
||||
if imported and (work["retrieved_at"] is None or imported < work["retrieved_at"]):
|
||||
work["retrieved_at"] = imported
|
||||
work["chunks"].append(
|
||||
{
|
||||
"chunk_id": str(artifact.provenance.get("chunk_id") or artifact.id),
|
||||
"digest": str(artifact.provenance.get("digest") or ""),
|
||||
"original_path": str(artifact.provenance.get("original_path") or ""),
|
||||
"path": artifact.path,
|
||||
}
|
||||
)
|
||||
chunk_to_work[artifact.id] = work["id"]
|
||||
works = list(grouped.values())
|
||||
for work in works:
|
||||
work["chunks"] = sorted(work["chunks"], key=lambda item: item["chunk_id"])
|
||||
return works, chunk_to_work
|
||||
|
||||
|
||||
def _project_entities(
|
||||
root: Path,
|
||||
slug: str,
|
||||
artifacts: list[KnowledgeArtifact],
|
||||
chunk_to_work: dict[str, str],
|
||||
works: list[dict[str, Any]],
|
||||
) -> tuple[list[dict[str, Any]], list[str]]:
|
||||
projected: list[dict[str, Any]] = []
|
||||
used_keys: set[tuple[str, str]] = set()
|
||||
for artifact in sorted(artifacts, key=lambda item: item.path):
|
||||
parsed = _parse_file(root / artifact.path)
|
||||
heading = _first_heading(parsed)
|
||||
if heading is None:
|
||||
raise InfospaceError(
|
||||
"invalid_entity_artifact",
|
||||
f"Entity has no heading: {artifact.id}",
|
||||
{"artifact_id": artifact.id},
|
||||
)
|
||||
title = display_title(heading)
|
||||
definition = extract_section_text(parsed, "Definition").strip()
|
||||
category = extract_section_text(parsed, "Category").strip()
|
||||
if not title or not definition or not category:
|
||||
raise InfospaceError(
|
||||
"invalid_entity_artifact",
|
||||
f"Entity is missing title, definition, or category: {artifact.id}",
|
||||
{"artifact_id": artifact.id},
|
||||
)
|
||||
kind_id = slugify(category)
|
||||
if not kind_id or kind_id in STORAGE_KINDS:
|
||||
raise InfospaceError(
|
||||
"invalid_presentation_kind",
|
||||
f"Category {category!r} is not a presentation kind",
|
||||
{"artifact_id": artifact.id, "category": category},
|
||||
)
|
||||
key = slugify(title)
|
||||
if not key:
|
||||
raise InfospaceError(
|
||||
"invalid_entity_key",
|
||||
f"Entity title does not yield a key: {title}",
|
||||
{"artifact_id": artifact.id},
|
||||
)
|
||||
suffix = 2
|
||||
base_key = key
|
||||
while (kind_id, key) in used_keys:
|
||||
key = f"{base_key}-{suffix}"
|
||||
suffix += 1
|
||||
used_keys.add((kind_id, key))
|
||||
source_ids = _source_refs(artifact, chunk_to_work, works)
|
||||
provenance = artifact.provenance
|
||||
projected.append(
|
||||
{
|
||||
"id": f"urn:ce:{slug}:{kind_id}:{key}",
|
||||
"kindId": kind_id,
|
||||
"title": title,
|
||||
"tldr": _tldr(definition),
|
||||
"summary": definition,
|
||||
"tags": [kind_id],
|
||||
"sourceRefs": source_ids,
|
||||
"properties": _entity_properties(artifact, provenance, category),
|
||||
"_heading": heading,
|
||||
}
|
||||
)
|
||||
kind_ids = sorted({item["kindId"] for item in projected})
|
||||
projected.sort(key=lambda item: item["id"])
|
||||
return projected, kind_ids
|
||||
|
||||
|
||||
def _project_relations(
|
||||
root: Path,
|
||||
slug: str,
|
||||
artifacts: list[KnowledgeArtifact],
|
||||
title_index: dict[str, list[dict[str, Any]]],
|
||||
chunk_to_work: dict[str, str],
|
||||
works: list[dict[str, Any]],
|
||||
) -> tuple[list[ResolvedRelation], list[QuarantinedRelation]]:
|
||||
resolved: list[ResolvedRelation] = []
|
||||
quarantined: list[QuarantinedRelation] = []
|
||||
accepted: list[tuple[KnowledgeArtifact, int, str, str, str, str, str, dict[str, Any], dict[str, Any]]] = []
|
||||
for artifact in sorted(artifacts, key=lambda item: item.path):
|
||||
text = (root / artifact.path).read_text(encoding="utf-8")
|
||||
for index, block in enumerate(_relation_blocks(text)):
|
||||
parsed = _parse_text(block, root / artifact.path)
|
||||
subject = extract_section_text(parsed, "Subject").strip()
|
||||
obj = extract_section_text(parsed, "Object").strip()
|
||||
relation_type = extract_section_text(parsed, "Relation Type").strip()
|
||||
predicate_text = extract_section_text(parsed, "Predicate").strip()
|
||||
if not subject or not obj or not relation_type:
|
||||
quarantined.append(
|
||||
QuarantinedRelation(
|
||||
artifact_id=artifact.id,
|
||||
block_index=index,
|
||||
subject=subject,
|
||||
object=obj,
|
||||
relation_type=relation_type,
|
||||
predicate_text=predicate_text,
|
||||
reason="missing_fields",
|
||||
)
|
||||
)
|
||||
continue
|
||||
left, left_reason = _resolve_endpoint(subject, title_index)
|
||||
right, right_reason = _resolve_endpoint(obj, title_index)
|
||||
if left is None or right is None:
|
||||
quarantined.append(
|
||||
QuarantinedRelation(
|
||||
artifact_id=artifact.id,
|
||||
block_index=index,
|
||||
subject=subject,
|
||||
object=obj,
|
||||
relation_type=relation_type,
|
||||
predicate_text=predicate_text,
|
||||
reason=_quarantine_reason(left_reason, right_reason),
|
||||
)
|
||||
)
|
||||
continue
|
||||
source_id = _source_refs(artifact, chunk_to_work, works)[0]
|
||||
accepted.append(
|
||||
(
|
||||
artifact,
|
||||
index,
|
||||
subject,
|
||||
obj,
|
||||
relation_type,
|
||||
predicate_text,
|
||||
source_id,
|
||||
left,
|
||||
right,
|
||||
)
|
||||
)
|
||||
|
||||
accepted.sort(key=lambda item: (item[0].id, item[1]))
|
||||
for number, item in enumerate(accepted, start=1):
|
||||
artifact, index, subject, obj, relation_type, predicate_text, source_id, left, right = item
|
||||
resolved.append(
|
||||
ResolvedRelation(
|
||||
artifact_id=artifact.id,
|
||||
block_index=index,
|
||||
subject=subject,
|
||||
object=obj,
|
||||
relation_type=relation_type,
|
||||
predicate_text=predicate_text,
|
||||
predicate=slugify(relation_type),
|
||||
from_id=left["id"],
|
||||
to_id=right["id"],
|
||||
relation_id=f"rel:{slug}:{number:04d}",
|
||||
source_id=source_id,
|
||||
)
|
||||
)
|
||||
quarantined.sort(key=lambda item: (item.artifact_id, item.block_index))
|
||||
return resolved, quarantined
|
||||
|
||||
|
||||
def _snapshot(
|
||||
*,
|
||||
slug: str,
|
||||
name: str,
|
||||
works: list[dict[str, Any]],
|
||||
entities: list[dict[str, Any]],
|
||||
resolved: list[ResolvedRelation],
|
||||
kind_ids: list[str],
|
||||
) -> dict[str, Any]:
|
||||
public_entities = []
|
||||
for entity in entities:
|
||||
public = {key: value for key, value in entity.items() if not key.startswith("_")}
|
||||
public_entities.append(public)
|
||||
return {
|
||||
"schemaVersion": SCHEMA_VERSION,
|
||||
"dataspace": {
|
||||
"id": slug,
|
||||
"title": name,
|
||||
"description": (
|
||||
f"Presentation of the {name} infospace ({slug}). "
|
||||
"Infospace-bench owns the artifacts. Content-explorer only presents them."
|
||||
),
|
||||
},
|
||||
"kinds": [
|
||||
{"id": kind_id, "label": _label(kind_id), "plural": _plural(kind_id)}
|
||||
for kind_id in kind_ids
|
||||
],
|
||||
"entities": public_entities,
|
||||
"relations": [
|
||||
{
|
||||
"id": item.relation_id,
|
||||
"predicate": item.predicate,
|
||||
"from": item.from_id,
|
||||
"to": item.to_id,
|
||||
"sourceRefs": [item.source_id],
|
||||
}
|
||||
for item in resolved
|
||||
],
|
||||
"sources": [_public_source(work) for work in works],
|
||||
"claims": [],
|
||||
}
|
||||
|
||||
|
||||
def _manifest(slug: str, name: str, snapshot: dict[str, Any]) -> dict[str, Any]:
|
||||
kind_ids = [item["id"] for item in snapshot["kinds"]]
|
||||
predicates = sorted({item["predicate"] for item in snapshot["relations"]})
|
||||
return {
|
||||
"schemaVersion": SCHEMA_VERSION,
|
||||
"id": slug,
|
||||
"home": {
|
||||
"heading": f"Explore {name}",
|
||||
"featuredKinds": list(kind_ids),
|
||||
},
|
||||
"navigation": [{"label": "Browse", "kinds": list(kind_ids)}],
|
||||
"kindViews": {
|
||||
kind_id: {"columns": ["name", "tldr", "tags"], "facets": ["tags"]}
|
||||
for kind_id in kind_ids
|
||||
},
|
||||
"panels": list(PANELS),
|
||||
"relationLabels": {predicate: _relation_wording(predicate) for predicate in predicates},
|
||||
"factPolicy": "show-all",
|
||||
}
|
||||
|
||||
|
||||
def _public_source(work: dict[str, Any]) -> dict[str, Any]:
|
||||
lines = []
|
||||
if work["creator"]:
|
||||
lines.append(f"creator: {work['creator']}")
|
||||
lines.append(f"identifier: {work['identifier']}")
|
||||
lines.append("provenance:")
|
||||
for chunk in work["chunks"]:
|
||||
lines.append(f"- chunk_id: {chunk['chunk_id']}")
|
||||
lines.append(f" digest: {chunk['digest']}")
|
||||
lines.append(f" original_path: {chunk['original_path']}")
|
||||
lines.append(f" path: {chunk['path']}")
|
||||
document: dict[str, Any] = {
|
||||
"id": work["id"],
|
||||
"uri": work["uri"],
|
||||
"title": work["title"],
|
||||
"license": work["license"],
|
||||
"attribution": "\n".join(lines),
|
||||
}
|
||||
if work["retrieved_at"]:
|
||||
document["retrievedAt"] = work["retrieved_at"]
|
||||
return document
|
||||
|
||||
|
||||
def _title_index(entities: list[dict[str, Any]]) -> dict[str, list[dict[str, Any]]]:
|
||||
index: dict[str, list[dict[str, Any]]] = {}
|
||||
for entity in entities:
|
||||
for value in (entity["title"], entity["_heading"]):
|
||||
key = slugify(value)
|
||||
if not key:
|
||||
continue
|
||||
bucket = index.setdefault(key, [])
|
||||
if all(existing["id"] != entity["id"] for existing in bucket):
|
||||
bucket.append(entity)
|
||||
return index
|
||||
|
||||
|
||||
def _resolve_endpoint(
|
||||
text: str,
|
||||
title_index: dict[str, list[dict[str, Any]]],
|
||||
) -> tuple[dict[str, Any] | None, str]:
|
||||
key = slugify(text)
|
||||
if not key:
|
||||
return None, "unresolved"
|
||||
matches = title_index.get(key) or []
|
||||
if len(matches) == 1:
|
||||
return matches[0], ""
|
||||
if len(matches) > 1:
|
||||
return None, "ambiguous"
|
||||
return None, "unresolved"
|
||||
|
||||
|
||||
def _quarantine_reason(left: str | None, right: str | None) -> str:
|
||||
reasons = [reason for reason in (left, right) if reason]
|
||||
if reasons == ["ambiguous"] or set(reasons) == {"ambiguous"}:
|
||||
if left == "ambiguous" and right == "ambiguous":
|
||||
return "ambiguous_endpoints"
|
||||
if left == "ambiguous":
|
||||
return "ambiguous_subject"
|
||||
return "ambiguous_object"
|
||||
if left and right:
|
||||
return "unresolved_endpoints"
|
||||
if left:
|
||||
return "unresolved_subject"
|
||||
return "unresolved_object"
|
||||
|
||||
|
||||
def _source_refs(
|
||||
artifact: KnowledgeArtifact,
|
||||
chunk_to_work: dict[str, str],
|
||||
works: list[dict[str, Any]],
|
||||
) -> list[str]:
|
||||
refs: list[str] = []
|
||||
for link in artifact.relationships:
|
||||
if link.get("type") != "generated_from":
|
||||
continue
|
||||
work_id = chunk_to_work.get(str(link.get("target") or ""))
|
||||
if work_id and work_id not in refs:
|
||||
refs.append(work_id)
|
||||
if refs:
|
||||
return refs
|
||||
if len(works) == 1:
|
||||
return [works[0]["id"]]
|
||||
raise InfospaceError(
|
||||
"unresolved_source_ref",
|
||||
f"No work-level source for {artifact.id}",
|
||||
{"artifact_id": artifact.id},
|
||||
)
|
||||
|
||||
|
||||
def _entity_properties(
|
||||
artifact: KnowledgeArtifact,
|
||||
provenance: dict[str, Any],
|
||||
category: str,
|
||||
) -> dict[str, str]:
|
||||
properties = {
|
||||
"category": category,
|
||||
"artifact_id": artifact.id,
|
||||
"artifact_path": artifact.path,
|
||||
}
|
||||
for key in ("workflow_id", "stage_id", "input_artifact_id", "provider"):
|
||||
value = provenance.get(key)
|
||||
if value:
|
||||
properties[key] = str(value)
|
||||
return properties
|
||||
|
||||
|
||||
def _relation_blocks(text: str) -> list[str]:
|
||||
blocks = []
|
||||
for part in RULE_SPLIT.split(text):
|
||||
if re.search(r"(?m)^##\s+Subject\s*$", part):
|
||||
blocks.append(part)
|
||||
return blocks
|
||||
|
||||
|
||||
def _parse_file(path: Path) -> ParsedMarkdownArtifact:
|
||||
return _parse_text(path.read_text(encoding="utf-8"), path)
|
||||
|
||||
|
||||
def _parse_text(text: str, path: Path) -> ParsedMarkdownArtifact:
|
||||
document = parse_markdown(text, source_path=str(path))
|
||||
return ParsedMarkdownArtifact(
|
||||
path=path,
|
||||
frontmatter=document.frontmatter,
|
||||
headings=document.headings,
|
||||
sections=document.sections,
|
||||
)
|
||||
|
||||
|
||||
def _first_heading(parsed: ParsedMarkdownArtifact) -> str | None:
|
||||
for heading in parsed.headings:
|
||||
if heading.level == 1:
|
||||
return heading.text
|
||||
return None
|
||||
|
||||
|
||||
def _tldr(definition: str) -> str:
|
||||
stripped = definition.strip()
|
||||
for mark in (". ", "? ", "! "):
|
||||
found = stripped.find(mark)
|
||||
if found != -1:
|
||||
return stripped[: found + 1]
|
||||
return stripped
|
||||
|
||||
|
||||
def _label(kind_id: str) -> str:
|
||||
return " ".join(part.capitalize() for part in kind_id.split("-"))
|
||||
|
||||
|
||||
def _plural(kind_id: str) -> str:
|
||||
label = _label(kind_id)
|
||||
if label.endswith("s"):
|
||||
return label
|
||||
return f"{label}s"
|
||||
|
||||
|
||||
def _relation_wording(predicate: str) -> dict[str, str]:
|
||||
preset = RELATION_LABELS.get(predicate)
|
||||
if preset:
|
||||
forward, inverse = preset
|
||||
return {"forward": forward, "inverse": inverse}
|
||||
words = " ".join(part.capitalize() for part in predicate.split("-"))
|
||||
return {"forward": words, "inverse": f"Inverse of {words}"}
|
||||
|
||||
|
||||
def _date(value: str) -> str | None:
|
||||
if len(value) >= 10 and value[4] == "-" and value[7] == "-":
|
||||
day = value[:10]
|
||||
if day[0:4].isdigit() and day[5:7].isdigit() and day[8:10].isdigit():
|
||||
return day
|
||||
return None
|
||||
|
||||
|
||||
def _write_json(path: Path, payload: dict[str, Any]) -> None:
|
||||
path.write_text(
|
||||
json.dumps(payload, indent=2, ensure_ascii=False, sort_keys=True) + "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
170
tests/test_content_explorer_projection.py
Normal file
170
tests/test_content_explorer_projection.py
Normal file
|
|
@ -0,0 +1,170 @@
|
|||
"""Projection of the Lefevre infospace into a content-explorer presentation."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import json
|
||||
|
||||
import yaml
|
||||
|
||||
from infospace_bench.markdown_adapter import extract_section_text, parse_markdown_artifact
|
||||
from infospace_bench.presentation import display_title, project_infospace
|
||||
from infospace_bench.semantics import slugify
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
LEFEVRE = ROOT / "infospaces" / "lefevre-reminiscences-of-a-stock-operator"
|
||||
CONTENT_EXPLORER = Path("/home/worsch/content-explorer")
|
||||
sys.path.insert(0, str(CONTENT_EXPLORER))
|
||||
|
||||
from content_explorer.contract import validate_documents # noqa: E402
|
||||
|
||||
|
||||
def test_lefevre_projection_matches_stored_artifacts_and_contract() -> None:
|
||||
first = project_infospace(LEFEVRE)
|
||||
second = project_infospace(LEFEVRE)
|
||||
snapshot = first.snapshot
|
||||
manifest = first.manifest
|
||||
|
||||
assert [item["id"] for item in snapshot["entities"]] == [
|
||||
item["id"] for item in second.snapshot["entities"]
|
||||
]
|
||||
assert [item["id"] for item in snapshot["relations"]] == [
|
||||
item["id"] for item in second.snapshot["relations"]
|
||||
]
|
||||
|
||||
index = yaml.safe_load((LEFEVRE / "artifacts" / "index.yaml").read_text(encoding="utf-8"))
|
||||
indexed = index["artifacts"]
|
||||
source_rows = [item for item in indexed if item["kind"] == "source"]
|
||||
entity_rows = [item for item in indexed if item["kind"] == "entity"]
|
||||
generated_rows = [item for item in indexed if item["kind"] == "generated"]
|
||||
relation_rows = [item for item in indexed if item["kind"] == "relation"]
|
||||
|
||||
assert len(entity_rows) == 14
|
||||
assert len(snapshot["entities"]) == 14
|
||||
assert {item["kindId"] for item in snapshot["entities"]} >= {
|
||||
"instrument",
|
||||
"strategy",
|
||||
"market",
|
||||
"event",
|
||||
"evidence-bearing-claim",
|
||||
"institution",
|
||||
"error",
|
||||
"trader",
|
||||
}
|
||||
assert {item["id"] for item in snapshot["kinds"]} == {
|
||||
item["kindId"] for item in snapshot["entities"]
|
||||
}
|
||||
assert {item["id"] for item in snapshot["kinds"]}.isdisjoint(
|
||||
{"source", "entity", "relation", "generated"}
|
||||
)
|
||||
|
||||
book = source_rows[0]["provenance"]["book_metadata"]
|
||||
assert len(snapshot["sources"]) == 1
|
||||
work = snapshot["sources"][0]
|
||||
assert work["license"] == book["rights"] == "Public domain in the USA."
|
||||
assert work["uri"] == book["source_url"]
|
||||
assert work["title"] == book["title"]
|
||||
attribution = work["attribution"]
|
||||
assert "provenance:" in attribution
|
||||
assert len(source_rows) == 6
|
||||
for source in source_rows:
|
||||
provenance = source["provenance"]
|
||||
assert provenance["digest"] in attribution
|
||||
assert provenance["original_path"] in attribution
|
||||
assert provenance["chunk_id"] in attribution
|
||||
|
||||
generated_titles = {item["title"] for item in generated_rows}
|
||||
generated_ids = {item["id"] for item in generated_rows}
|
||||
by_path = {item["properties"]["artifact_path"]: item for item in snapshot["entities"]}
|
||||
assert len(by_path) == 14
|
||||
for row in entity_rows:
|
||||
projected = by_path[row["path"]]
|
||||
parsed = parse_markdown_artifact(LEFEVRE / row["path"])
|
||||
heading = next(item.text for item in parsed.headings if item.level == 1)
|
||||
definition = extract_section_text(parsed, "Definition").strip()
|
||||
category = extract_section_text(parsed, "Category").strip()
|
||||
assert projected["title"] == display_title(heading)
|
||||
assert projected["summary"] == definition
|
||||
assert projected["tldr"]
|
||||
assert definition.startswith(projected["tldr"])
|
||||
assert projected["kindId"] == slugify(category)
|
||||
assert projected["id"] == (
|
||||
f"urn:ce:lefevre-reminiscences-of-a-stock-operator:"
|
||||
f"{projected['kindId']}:{slugify(projected['title'])}"
|
||||
)
|
||||
assert projected["sourceRefs"] == [work["id"]]
|
||||
assert projected["properties"]["workflow_id"] == row["provenance"]["workflow_id"]
|
||||
assert projected["properties"]["input_artifact_id"] == row["provenance"]["input_artifact_id"]
|
||||
assert projected["title"] not in generated_titles
|
||||
assert projected["id"] not in generated_ids
|
||||
|
||||
block_count = 0
|
||||
for row in relation_rows:
|
||||
text = (LEFEVRE / row["path"]).read_text(encoding="utf-8")
|
||||
block_count += text.count("\n## Subject") + int(text.startswith("## Subject"))
|
||||
assert block_count == 27
|
||||
assert len(first.resolved_relations) + len(first.quarantined_relations) == block_count
|
||||
assert first.quarantined_relations
|
||||
assert len(snapshot["relations"]) == len(first.resolved_relations)
|
||||
entities_by_id = {item["id"]: item for item in snapshot["entities"]}
|
||||
relations_by_id = {item["id"]: item for item in snapshot["relations"]}
|
||||
for record in first.resolved_relations:
|
||||
projected = relations_by_id[record.relation_id]
|
||||
assert projected["predicate"] == record.relation_type.replace("_", "-")
|
||||
assert " " not in projected["predicate"]
|
||||
if record.predicate_text != projected["predicate"]:
|
||||
assert projected["predicate"] != record.predicate_text
|
||||
assert slugify(record.subject) == slugify(entities_by_id[record.from_id]["title"])
|
||||
assert slugify(record.object) == slugify(entities_by_id[record.to_id]["title"])
|
||||
assert projected["from"] == record.from_id
|
||||
assert projected["to"] == record.to_id
|
||||
assert projected["sourceRefs"] == [work["id"]]
|
||||
resolved_keys = {(item.artifact_id, item.block_index) for item in first.resolved_relations}
|
||||
for record in first.quarantined_relations:
|
||||
assert (record.artifact_id, record.block_index) not in resolved_keys
|
||||
subject_hits = [
|
||||
item
|
||||
for item in snapshot["entities"]
|
||||
if slugify(item["title"]) == slugify(record.subject)
|
||||
]
|
||||
object_hits = [
|
||||
item
|
||||
for item in snapshot["entities"]
|
||||
if slugify(item["title"]) == slugify(record.object)
|
||||
]
|
||||
assert not (len(subject_hits) == 1 and len(object_hits) == 1)
|
||||
|
||||
assert manifest["schemaVersion"] == "0.1"
|
||||
assert manifest["id"] == "lefevre-reminiscences-of-a-stock-operator"
|
||||
assert manifest["panels"] == ["summary", "relations", "evidence", "claims"]
|
||||
assert manifest["factPolicy"] == "show-all"
|
||||
assert set(manifest["kindViews"]) == {item["id"] for item in snapshot["kinds"]}
|
||||
for view in manifest["kindViews"].values():
|
||||
assert view["columns"]
|
||||
assert "facets" in view
|
||||
assert set(manifest["home"]["featuredKinds"]) == {item["id"] for item in snapshot["kinds"]}
|
||||
assert set(manifest["relationLabels"]) == {item["predicate"] for item in snapshot["relations"]}
|
||||
for wording in manifest["relationLabels"].values():
|
||||
assert wording["forward"]
|
||||
assert wording["inverse"]
|
||||
|
||||
export_dir = LEFEVRE / "exports" / "content-explorer"
|
||||
assert json.loads((export_dir / "snapshot.json").read_text(encoding="utf-8")) == snapshot
|
||||
assert json.loads((export_dir / "manifest.json").read_text(encoding="utf-8")) == manifest
|
||||
|
||||
errors = validate_documents(snapshot, manifest, require_size=True)
|
||||
assert errors == []
|
||||
raw_errors = validate_documents(index, index, require_size=True)
|
||||
assert raw_errors
|
||||
|
||||
note = (LEFEVRE / "exports" / "content-explorer" / "PROJECTION.md").read_text(encoding="utf-8")
|
||||
assert "lefevre-reminiscences-of-a-stock-operator" in note
|
||||
assert "content-explorer only presents" in note.lower()
|
||||
assert "infospace-bench still owns" in note.lower()
|
||||
|
||||
print("ENTITY_IDS " + ",".join(item["id"] for item in snapshot["entities"]))
|
||||
print("RELATION_IDS " + ",".join(item["id"] for item in snapshot["relations"]))
|
||||
print("VALIDATOR_ERRORS 0")
|
||||
print("RAW_ARTIFACTS_ERRORS " + str(len(raw_errors)))
|
||||
|
|
@ -4,7 +4,7 @@ type: workplan
|
|||
title: "Bind infospaces to content-explorer as the presentation UI"
|
||||
domain: communication
|
||||
repo: infospace-bench
|
||||
status: ready
|
||||
status: finished
|
||||
owner: grok
|
||||
topic_slug: content-explorer-presentation
|
||||
created: "2026-10-14"
|
||||
|
|
@ -137,13 +137,21 @@ do not edit `/home/worsch/content-explorer`.
|
|||
- No binding of any infospace other than by specifying this reusable shape
|
||||
and carrying it out on the Lefevre example.
|
||||
|
||||
## Outcome
|
||||
|
||||
`infospace_bench.presentation.project_infospace` projects an infospace to one
|
||||
snapshot and one manifest. The Lefevre result is
|
||||
`infospaces/lefevre-reminiscences-of-a-stock-operator/exports/content-explorer/`.
|
||||
`PROJECTION.md` there records the reusable rule. Lefevre artifact files and
|
||||
the content-explorer application were not edited.
|
||||
|
||||
## Tasks
|
||||
|
||||
### T01 — Specify the reusable projection rule
|
||||
|
||||
```task
|
||||
id: IB-WP-0022-T01
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "128d6123-f855-529a-bf8f-7eb7442872cd"
|
||||
```
|
||||
|
|
@ -163,7 +171,7 @@ that Lefevre is the example. Do not treat storage kinds `source`, `entity`,
|
|||
|
||||
```task
|
||||
id: IB-WP-0022-T02
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "5861057c-865c-5c4c-8537-9b9a93bda503"
|
||||
```
|
||||
|
|
@ -180,7 +188,7 @@ the browse set. Do not rewrite the infospace.
|
|||
|
||||
```task
|
||||
id: IB-WP-0022-T03
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "922bddc1-ae21-523d-ad60-de1e4536954e"
|
||||
```
|
||||
|
|
@ -196,7 +204,7 @@ a `factPolicy`. Presentation uses this manifest. It does not add UI code.
|
|||
|
||||
```task
|
||||
id: IB-WP-0022-T04
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "f85eae30-b72c-5297-b9cf-d4376b13faa2"
|
||||
```
|
||||
|
|
@ -211,7 +219,7 @@ treats the current `artifacts/` tree as an already-valid snapshot.
|
|||
|
||||
```task
|
||||
id: IB-WP-0022-T05
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "3852fcf3-d565-592d-8f18-6793948d7473"
|
||||
```
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue