tegwick
48af153ce7
Assistant: codex Assistant-Model: gpt-6-astra Assistant-Session: 01a0e8cd-2bda-7890-a3ff-7bec606cdf51
267 lines
5.4 KiB
Markdown
267 lines
5.4 KiB
Markdown
# Research Signals
|
|
|
|
## Purpose
|
|
|
|
Coulomb Research needs observable signals that help participants and agents understand what is attracting attention, shaping downstream work, surviving criticism, or generating useful activity.
|
|
|
|
Signals are measurements of the research system.
|
|
|
|
They are not truth.
|
|
|
|
## 1. Foundational Separation
|
|
|
|
Coulomb should keep at least four concepts distinct:
|
|
|
|
```text
|
|
ATTENTION
|
|
What are people looking at?
|
|
|
|
INFLUENCE
|
|
What does other work depend on?
|
|
|
|
EPISTEMIC SUPPORT
|
|
What survives available evidence and criticism?
|
|
|
|
UTILITY
|
|
What is being reused?
|
|
```
|
|
|
|
A fifth signal family is especially important:
|
|
|
|
```text
|
|
GENERATIVITY
|
|
What produces valuable new questions, methods, experiments, or explanations?
|
|
```
|
|
|
|
No single signal should silently stand in for the others.
|
|
|
|
## 2. Signal Families
|
|
|
|
### 2.1 Attention
|
|
|
|
Possible observations:
|
|
|
|
- views,
|
|
- reads,
|
|
- downloads,
|
|
- bookmarks,
|
|
- external mentions,
|
|
- discussion activity,
|
|
- watch/follow events.
|
|
|
|
Attention may be positive, negative, or merely fashionable.
|
|
|
|
It must not be interpreted as quality.
|
|
|
|
### 2.2 Influence
|
|
|
|
Possible observations:
|
|
|
|
- conventional citations,
|
|
- typed citations,
|
|
- propositions depending on the artifact,
|
|
- methods reused downstream,
|
|
- later work extending the artifact,
|
|
- inclusion in syntheses,
|
|
- downstream research programs.
|
|
|
|
Typed relations are preferable to raw citation count.
|
|
|
|
Example:
|
|
|
|
```text
|
|
depends-on
|
|
extends
|
|
supports
|
|
challenges
|
|
uses-method
|
|
uses-data
|
|
uses-software
|
|
```
|
|
|
|
### 2.3 Epistemic Support
|
|
|
|
Possible observations:
|
|
|
|
- independent evidence,
|
|
- successful replication,
|
|
- prediction success,
|
|
- unresolved criticism,
|
|
- contradictory evidence,
|
|
- retractions or supersession,
|
|
- methodological robustness.
|
|
|
|
Support should usually be represented as a multi-dimensional state rather than one score.
|
|
|
|
### 2.4 Utility and Reuse
|
|
|
|
Possible observations:
|
|
|
|
- software imports,
|
|
- dataset reuse,
|
|
- benchmark reuse,
|
|
- method adoption,
|
|
- protocol adoption,
|
|
- implementation reuse,
|
|
- educational reuse,
|
|
- operational deployment.
|
|
|
|
Reuse may reveal value that citation systems miss.
|
|
|
|
### 2.5 Criticism
|
|
|
|
Possible observations:
|
|
|
|
- substantive critiques,
|
|
- counterexamples,
|
|
- failed replications,
|
|
- methodological objections,
|
|
- conceptual contradictions,
|
|
- corrections.
|
|
|
|
A high criticism signal is not inherently negative.
|
|
|
|
It may indicate that an artifact is important enough to test aggressively.
|
|
|
|
### 2.6 Generativity
|
|
|
|
Possible observations:
|
|
|
|
- new questions spawned,
|
|
- new propositions created,
|
|
- new experiments triggered,
|
|
- research programs formed,
|
|
- software or datasets produced,
|
|
- cross-domain extensions,
|
|
- independent follow-up work.
|
|
|
|
Generativity measures whether research creates productive new problem spaces.
|
|
|
|
### 2.7 Velocity
|
|
|
|
Signals should often be time-normalized.
|
|
|
|
Examples:
|
|
|
|
```text
|
|
citations / month
|
|
reuses / month
|
|
replications / year
|
|
new dependent propositions / week
|
|
```
|
|
|
|
Velocity helps surface emerging work before accumulated totals dominate.
|
|
|
|
## 3. Scope and Normalization
|
|
|
|
Signals should support explicit comparison scopes.
|
|
|
|
Possible dimensions:
|
|
|
|
- field,
|
|
- subfield,
|
|
- year,
|
|
- publication age,
|
|
- artifact type,
|
|
- cohort,
|
|
- geography,
|
|
- research program.
|
|
|
|
Comparing raw totals across biology, mathematics, AI, and philosophy is usually misleading.
|
|
|
|
Percentiles are often preferable to universal thresholds.
|
|
|
|
## 4. Signal Representation
|
|
|
|
A research artifact should expose a signal vector rather than a universal score.
|
|
|
|
Example:
|
|
|
|
```text
|
|
Attention 98th percentile
|
|
Influence 91st percentile
|
|
Reuse 84th percentile
|
|
Replication 62nd percentile
|
|
Criticism 77th percentile
|
|
Generativity 95th percentile
|
|
Support contested
|
|
```
|
|
|
|
The exact metrics and normalization method must remain inspectable.
|
|
|
|
## 5. No Canonical Impact Score
|
|
|
|
Coulomb should avoid a single global "impact score".
|
|
|
|
Reasons:
|
|
|
|
- it collapses unlike dimensions,
|
|
- it invites gaming,
|
|
- it hides value judgments,
|
|
- it creates prestige hierarchies,
|
|
- it becomes a Goodhart target,
|
|
- it mistakes observability for truth.
|
|
|
|
If composite scores are offered, the weighting function must be explicit and contestable.
|
|
|
|
## 6. Signals as Control Inputs
|
|
|
|
Signals can drive research-system behavior.
|
|
|
|
Examples:
|
|
|
|
```text
|
|
high attention
|
|
→ allocate additional criticism
|
|
|
|
rapid reuse
|
|
→ prioritize replication
|
|
|
|
foundational dependency
|
|
→ audit provenance and robustness
|
|
|
|
high controversy
|
|
→ synthesize competing explanations
|
|
|
|
high generativity
|
|
→ inspect what made the work productive
|
|
```
|
|
|
|
Signals therefore participate in the cybernetic feedback loop.
|
|
|
|
## 7. Actor Signals
|
|
|
|
Actor-level summaries should be derived from contribution events, not maintained as universal status.
|
|
|
|
Examples:
|
|
|
|
- replication history,
|
|
- prediction history,
|
|
- criticism history,
|
|
- artifact reuse,
|
|
- successful resource allocation,
|
|
- correction acceptance.
|
|
|
|
Different communities should be able to derive different reputation functions.
|
|
|
|
## 8. Anti-Gaming Principles
|
|
|
|
Signals should:
|
|
|
|
- rely on attributable events,
|
|
- discount obvious self-interaction where appropriate,
|
|
- distinguish automated from human activity,
|
|
- expose suspicious activity,
|
|
- use time windows,
|
|
- use field normalization,
|
|
- preserve raw events for independent recomputation.
|
|
|
|
Whenever possible, Coulomb should publish enough event data for alternative ranking functions to be constructed independently.
|
|
|
|
## 9. Research Question
|
|
|
|
The signal system itself belongs to RP-000.
|
|
|
|
Key question:
|
|
|
|
> Which observable signals best predict durable explanatory progress without creating destructive incentives?
|