Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Assistant: claude-code Assistant-Model: opus Assistant-Process: 1370019@bnt-lap001 Assistant-Session: d462dd63-16de-4189-ae70-b9cd6566bf87
1610 lines
29 KiB
Markdown
Executable file
1610 lines
29 KiB
Markdown
Executable file
# plenty-gobble — Use Case Catalog
|
|
|
|
**Version:** 0.1
|
|
**Date:** 2026-09-12
|
|
**Status:** Initial use-case catalog
|
|
**Derived from:** `2026-09-12-IncidentMetricsPrinciples-Agentic-Observability-Research.md`
|
|
**Repository:** `plenty-gobble`
|
|
|
|
---
|
|
|
|
# 1. Purpose
|
|
|
|
This catalog translates the principles of evidence-preserving agentic observability into concrete use cases for `plenty-gobble`.
|
|
|
|
The primary design premise is:
|
|
|
|
> **Store broadly, query selectively.**
|
|
|
|
The primary epistemic constraint is:
|
|
|
|
> **An agent must never mistake unavailable evidence for negative evidence.**
|
|
|
|
Accordingly, the use cases below do not only describe what data should be queryable. They also describe:
|
|
|
|
- what evidence must remain available;
|
|
- what loss mechanisms may affect the result;
|
|
- what completeness metadata must be exposed;
|
|
- how agents should reason about confidence;
|
|
- which `plenty-gobble` capabilities are required.
|
|
|
|
---
|
|
|
|
# 2. Catalog Dimensions
|
|
|
|
Each use case is described using the following dimensions.
|
|
|
|
## Actor
|
|
|
|
Who initiates or consumes the use case.
|
|
|
|
Typical actors:
|
|
|
|
- human operator;
|
|
- incident commander;
|
|
- SRE;
|
|
- developer;
|
|
- product engineer;
|
|
- security analyst;
|
|
- autonomous analysis agent;
|
|
- coding agent;
|
|
- supervisory agent.
|
|
|
|
## Goal
|
|
|
|
What the actor is trying to determine or accomplish.
|
|
|
|
## Evidence
|
|
|
|
What telemetry or historical context is required.
|
|
|
|
## Risk of Incompleteness
|
|
|
|
How sampling, retention, filtering, rollups, or cardinality reduction could lead to a false or overconfident conclusion.
|
|
|
|
## `plenty-gobble` Capabilities
|
|
|
|
Which platform capabilities are required.
|
|
|
|
## Expected Output
|
|
|
|
What a successful result should look like.
|
|
|
|
---
|
|
|
|
# 3. Capability Vocabulary
|
|
|
|
The use cases refer to the following recurring `plenty-gobble` capabilities.
|
|
|
|
### C1 — Full-Fidelity Telemetry Retention
|
|
|
|
Retain raw or logically lossless logs, traces, metrics-derived events, and related observations where economically feasible.
|
|
|
|
### C2 — Hot/Cold Tiering
|
|
|
|
Move older data to slower, cheaper storage without making it logically unavailable.
|
|
|
|
### C3 — Loss Manifest
|
|
|
|
Record sampling, filtering, rollups, attribute removal, retention gaps, cardinality caps, deduplication, and similar lossy transformations.
|
|
|
|
### C4 — Evidence Completeness API
|
|
|
|
Expose coverage, fidelity, known gaps, transformations, retained dimensions, and missing periods for a given query context.
|
|
|
|
### C5 — Case File
|
|
|
|
Create a bounded incident evidence package containing:
|
|
|
|
- observation;
|
|
- title;
|
|
- comments;
|
|
- traces;
|
|
- metrics;
|
|
- logs;
|
|
- prior incidents;
|
|
- evidence completeness;
|
|
- resulting analysis.
|
|
|
|
### C6 — Incident Archive
|
|
|
|
Persist Case Files, postmortems, root causes, blast radius, remediation, and precedent relationships beyond raw telemetry retention.
|
|
|
|
### C7 — Adaptive Query-Time Pruning
|
|
|
|
Use segment, row-group, field, and synopsis metadata to avoid reading irrelevant data without destroying retained meaning.
|
|
|
|
### C8 — Cross-Signal Correlation
|
|
|
|
Correlate logs, traces, metrics, events, deployment state, and related metadata across a shared time and identity model.
|
|
|
|
### C9 — Evidence Fidelity Classification
|
|
|
|
Expose whether data is lossless, sampled, aggregated, filtered, partially retained, or of unknown provenance.
|
|
|
|
### C10 — Query Telemetry
|
|
|
|
Record what evidence a query considered, skipped, opened, and could not access.
|
|
|
|
### C11 — Prior-Incident Retrieval
|
|
|
|
Retrieve semantically and structurally relevant historical incidents.
|
|
|
|
### C12 — Full-Fidelity Reference Samples
|
|
|
|
Preserve selected complete traffic windows or cohorts where the full population cannot be retained.
|
|
|
|
### C13 — Multi-Tenant Isolation
|
|
|
|
Keep telemetry, metadata, query state, incident records, and optimization policies tenant-safe.
|
|
|
|
### C14 — Evidence-Aware Confidence
|
|
|
|
Allow downstream agents to distinguish:
|
|
|
|
- reasoning confidence;
|
|
- evidence completeness;
|
|
- evidence reliability.
|
|
|
|
---
|
|
|
|
# 4. Use Case Index
|
|
|
|
| ID | Use Case | Primary Actor | Core Value |
|
|
|---|---|---|---|
|
|
| UC-01 | Unknown-Cause Incident Investigation | Agent / SRE | Diagnose incidents without assuming future questions |
|
|
| UC-02 | Evidence Completeness Check | Agent / Operator | Determine whether absence of evidence is meaningful |
|
|
| UC-03 | Historical Precedent Search | Agent / SRE | Find prior incidents with similar symptoms |
|
|
| UC-04 | Blast-Radius Determination | Incident Commander / Agent | Identify all affected services, tenants, regions, builds |
|
|
| UC-05 | Root-Cause Evidence Package | Agent / Human Reviewer | Produce an auditable Case File |
|
|
| UC-06 | Cold-Tier Historical Investigation | Agent | Reach old evidence without keeping it on expensive hot storage |
|
|
| UC-07 | Sampled-Telemetry Interpretation | Agent / SRE | Reason correctly from incomplete populations |
|
|
| UC-08 | Rollup-Aware Analysis | Agent / Analyst | Know which dimensions disappeared during aggregation |
|
|
| UC-09 | Edge-Dedup Incident Reconstruction | Agent | Use semantically compacted repeated events without losing counts |
|
|
| UC-10 | Severity-Filter Risk Detection | Agent / Platform Engineer | Detect when low-severity context may have been discarded |
|
|
| UC-11 | Long-Term Regression Analysis | Developer / Agent | Compare incidents across releases and time |
|
|
| UC-12 | Build/Deployment Correlation | Developer / Agent | Relate failures to builds, versions, deployments, flags |
|
|
| UC-13 | Rare Failure Discovery | Agent / SRE | Find low-frequency failure patterns hidden by averages |
|
|
| UC-14 | Trace-to-Log-to-Metric Correlation | Agent | Traverse evidence across observability signal types |
|
|
| UC-15 | Agent Self-Check for Missing Evidence | Analysis Agent | Challenge its own conclusions when context is incomplete |
|
|
| UC-16 | Postmortem as Durable Organizational Memory | Agent / Team | Preserve causal knowledge after raw telemetry expires |
|
|
| UC-17 | Full-Fidelity Calibration Cohort | Evaluation Agent | Compare sampled data against complete reference data |
|
|
| UC-18 | Telemetry Policy Impact Assessment | Platform Team | Measure the operational effect of sampling/retention changes |
|
|
| UC-19 | Evidence-Preserving Cost Optimization | Platform Team | Reduce storage cost without silently destroying future questions |
|
|
| UC-20 | Agentic Coding Failure Analysis | Coding Agent / Platform Team | Evaluate coding-agent behavior using complete operational evidence |
|
|
| UC-21 | Security / Audit Investigation | Security Analyst / Agent | Preserve event chains and expose missing evidence |
|
|
| UC-22 | Customer-Specific Incident Analysis | Support / Agent | Isolate customer impact and precedent safely |
|
|
| UC-23 | Regional Incident Comparison | Operator / Agent | Compare behavior across regions without rollup blindness |
|
|
| UC-24 | Incident Case Handoff | Human ↔ Agent | Transfer bounded evidence and its limitations explicitly |
|
|
| UC-25 | Confidence-Gated Automated Remediation | Supervisory Agent | Prevent high-risk actions when evidence coverage is insufficient |
|
|
|
|
---
|
|
|
|
# 5. UC-01 — Unknown-Cause Incident Investigation
|
|
|
|
## Actor
|
|
|
|
Autonomous incident-analysis agent, SRE, developer.
|
|
|
|
## Goal
|
|
|
|
Determine the root cause of an incident when the relevant hypothesis is not known in advance.
|
|
|
|
## Trigger
|
|
|
|
Examples:
|
|
|
|
```text
|
|
latency spike
|
|
error-rate increase
|
|
customer complaint
|
|
unexpected restart
|
|
queue buildup
|
|
partial service degradation
|
|
```
|
|
|
|
## Evidence
|
|
|
|
Potentially:
|
|
|
|
- complete logs;
|
|
- traces;
|
|
- metrics;
|
|
- resource attributes;
|
|
- deployment metadata;
|
|
- prior incidents;
|
|
- configuration changes;
|
|
- topology;
|
|
- tenant/customer dimensions.
|
|
|
|
## Risk of Incompleteness
|
|
|
|
The decisive signal may exist in:
|
|
|
|
- low-severity logs;
|
|
- a rare trace;
|
|
- a dimension previously considered high-cardinality;
|
|
- a time period outside normal retention;
|
|
- an event removed by sampling;
|
|
- a field not included in a rollup.
|
|
|
|
## Required Capabilities
|
|
|
|
C1, C2, C4, C5, C7, C8, C11, C14.
|
|
|
|
## Expected Output
|
|
|
|
```text
|
|
root cause
|
|
supporting evidence
|
|
contradicting evidence
|
|
blast radius
|
|
precedent
|
|
evidence completeness
|
|
reasoning confidence
|
|
```
|
|
|
|
## Design Note
|
|
|
|
This is the canonical use case for the `plenty-gobble` thesis.
|
|
|
|
The system should assume that the important question will often be discovered only after investigation begins.
|
|
|
|
---
|
|
|
|
# 6. UC-02 — Evidence Completeness Check
|
|
|
|
## Actor
|
|
|
|
Agent or human investigator.
|
|
|
|
## Goal
|
|
|
|
Determine whether available evidence is sufficient to support a conclusion.
|
|
|
|
## Example Question
|
|
|
|
```text
|
|
"We found no failed login events. Does that mean no failed login occurred?"
|
|
```
|
|
|
|
## Evidence Required
|
|
|
|
Metadata describing:
|
|
|
|
- collection coverage;
|
|
- sampling;
|
|
- filtering;
|
|
- retention;
|
|
- missing time windows;
|
|
- cardinality caps;
|
|
- rollup dimensions;
|
|
- source health.
|
|
|
|
## Risk of Incompleteness
|
|
|
|
Without explicit completeness information:
|
|
|
|
```text
|
|
no retained evidence
|
|
```
|
|
|
|
may be interpreted as:
|
|
|
|
```text
|
|
evidence of absence
|
|
```
|
|
|
|
## Required Capabilities
|
|
|
|
C3, C4, C9, C10, C14.
|
|
|
|
## Expected Output
|
|
|
|
Example:
|
|
|
|
```text
|
|
Observation:
|
|
No failed-login events are present.
|
|
|
|
Coverage:
|
|
62%
|
|
|
|
Known loss:
|
|
Authentication logs sampled at 10% for this interval.
|
|
|
|
Conclusion:
|
|
Absence is not strong evidence that no failures occurred.
|
|
```
|
|
|
|
---
|
|
|
|
# 7. UC-03 — Historical Precedent Search
|
|
|
|
## Actor
|
|
|
|
Incident-analysis agent or SRE.
|
|
|
|
## Goal
|
|
|
|
Determine whether the current situation resembles a previous incident.
|
|
|
|
## Inputs
|
|
|
|
- symptoms;
|
|
- service names;
|
|
- metric anomalies;
|
|
- log patterns;
|
|
- trace behavior;
|
|
- deployment context;
|
|
- affected customers or regions.
|
|
|
|
## Evidence
|
|
|
|
- historical Case Files;
|
|
- postmortems;
|
|
- retained raw telemetry where available;
|
|
- causal chains;
|
|
- root causes;
|
|
- remediation records.
|
|
|
|
## Required Capabilities
|
|
|
|
C6, C11, C2.
|
|
|
|
## Expected Output
|
|
|
|
```text
|
|
possible precedents
|
|
similarity rationale
|
|
matching symptoms
|
|
differing symptoms
|
|
previous root cause
|
|
previous remediation
|
|
evidence available for comparison
|
|
```
|
|
|
|
## Design Note
|
|
|
|
Incident memory should be queryable as structured evidence, not only as prose documents.
|
|
|
|
---
|
|
|
|
# 8. UC-04 — Blast-Radius Determination
|
|
|
|
## Actor
|
|
|
|
Incident commander, support team, autonomous incident agent.
|
|
|
|
## Goal
|
|
|
|
Determine what is affected.
|
|
|
|
Typical dimensions:
|
|
|
|
```text
|
|
services
|
|
customers
|
|
regions
|
|
builds
|
|
clusters
|
|
hosts
|
|
versions
|
|
tenants
|
|
APIs
|
|
dependencies
|
|
```
|
|
|
|
## Risk of Incompleteness
|
|
|
|
Cardinality limits and rollups may have removed exactly the dimensions needed to determine scope.
|
|
|
|
## Required Capabilities
|
|
|
|
C1, C4, C8, C9.
|
|
|
|
## Expected Output
|
|
|
|
```text
|
|
affected entities
|
|
likely unaffected entities
|
|
uncertain entities
|
|
coverage by dimension
|
|
```
|
|
|
|
Example:
|
|
|
|
```text
|
|
Customer:
|
|
41 confirmed affected
|
|
2,118 confirmed unaffected
|
|
893 indeterminate because customer_id was removed from rolled-up metrics
|
|
```
|
|
|
|
---
|
|
|
|
# 9. UC-05 — Root-Cause Evidence Package
|
|
|
|
## Actor
|
|
|
|
Analysis agent preparing a result for human review.
|
|
|
|
## Goal
|
|
|
|
Create an auditable incident Case File.
|
|
|
|
## Contents
|
|
|
|
### Input
|
|
|
|
- Observation
|
|
- Title
|
|
- Comments
|
|
- Traces
|
|
- Metrics
|
|
- Logs
|
|
- Prior Incidents
|
|
|
|
### Evidence Envelope
|
|
|
|
- coverage;
|
|
- fidelity;
|
|
- known gaps;
|
|
- loss manifests;
|
|
- retention boundaries;
|
|
- collection failures.
|
|
|
|
### Output
|
|
|
|
- Root Cause
|
|
- Evidence
|
|
- Precedent
|
|
- Blast Radius
|
|
- Confidence
|
|
|
|
## Required Capabilities
|
|
|
|
C3, C4, C5, C6, C14.
|
|
|
|
## Expected Output
|
|
|
|
A portable Case File that another human or agent can independently inspect.
|
|
|
|
---
|
|
|
|
# 10. UC-06 — Cold-Tier Historical Investigation
|
|
|
|
## Actor
|
|
|
|
Autonomous agent.
|
|
|
|
## Goal
|
|
|
|
Investigate telemetry months or years old.
|
|
|
|
## Motivation
|
|
|
|
An agent can tolerate slower retrieval if the data remains available.
|
|
|
|
## Example
|
|
|
|
```text
|
|
"Did this exact failure mode occur before the migration six months ago?"
|
|
```
|
|
|
|
## Required Capabilities
|
|
|
|
C1, C2, C7, C11.
|
|
|
|
## Expected Behavior
|
|
|
|
The platform may respond:
|
|
|
|
```text
|
|
recent data < 1 second
|
|
90-day data several seconds
|
|
12-month archive tens of seconds
|
|
```
|
|
|
|
without changing query semantics.
|
|
|
|
## Design Principle
|
|
|
|
> Cold and slow is better than deleted.
|
|
|
|
---
|
|
|
|
# 11. UC-07 — Sampled-Telemetry Interpretation
|
|
|
|
## Actor
|
|
|
|
Agent or analyst.
|
|
|
|
## Goal
|
|
|
|
Use sampled data without confusing the retained population with the actual population.
|
|
|
|
## Required Metadata
|
|
|
|
```text
|
|
input count
|
|
output count
|
|
sampling rate
|
|
sampling algorithm
|
|
per-class sampling rates
|
|
biases
|
|
retained reference sample
|
|
```
|
|
|
|
## Example
|
|
|
|
```text
|
|
input traffic: 10,000,000
|
|
normal retained: 100,000
|
|
errors retained: 23,411
|
|
|
|
normal sampling: 1%
|
|
error sampling: 100%
|
|
```
|
|
|
|
## Required Capabilities
|
|
|
|
C3, C4, C9, C12.
|
|
|
|
## Expected Output
|
|
|
|
The agent should distinguish observed frequency from inferred population frequency.
|
|
|
|
---
|
|
|
|
# 12. UC-08 — Rollup-Aware Analysis
|
|
|
|
## Actor
|
|
|
|
Agent, analyst.
|
|
|
|
## Goal
|
|
|
|
Reason over rolled-up telemetry while knowing what dimensions can no longer be reconstructed.
|
|
|
|
## Example
|
|
|
|
A metric remains grouped by:
|
|
|
|
```text
|
|
region
|
|
build
|
|
```
|
|
|
|
but no longer by:
|
|
|
|
```text
|
|
customer
|
|
instance
|
|
request_type
|
|
```
|
|
|
|
## Required Capabilities
|
|
|
|
C3, C4, C9.
|
|
|
|
## Expected Output
|
|
|
|
```text
|
|
Question:
|
|
Was customer X uniquely affected?
|
|
|
|
Answer:
|
|
Cannot be established from this rollup.
|
|
customer_id was removed before aggregation.
|
|
```
|
|
|
|
---
|
|
|
|
# 13. UC-09 — Edge-Dedup Incident Reconstruction
|
|
|
|
## Actor
|
|
|
|
Agent or SRE.
|
|
|
|
## Goal
|
|
|
|
Reconstruct an event sequence after repetitive events have been semantically deduplicated.
|
|
|
|
## Preferred Representation
|
|
|
|
```text
|
|
event fingerprint
|
|
count
|
|
first_seen
|
|
last_seen
|
|
representative payload
|
|
```
|
|
|
|
Potential additional data:
|
|
|
|
```text
|
|
inter-arrival sketch
|
|
periodicity
|
|
distribution
|
|
```
|
|
|
|
## Required Capabilities
|
|
|
|
C3, C4, C9.
|
|
|
|
## Design Principle
|
|
|
|
Prefer:
|
|
|
|
```text
|
|
semantic compaction
|
|
```
|
|
|
|
over:
|
|
|
|
```text
|
|
blind deletion
|
|
```
|
|
|
|
---
|
|
|
|
# 14. UC-10 — Severity-Filter Risk Detection
|
|
|
|
## Actor
|
|
|
|
Agent or telemetry platform engineer.
|
|
|
|
## Goal
|
|
|
|
Detect when incident context may have been removed by severity filtering.
|
|
|
|
## Example
|
|
|
|
```text
|
|
ERROR retained
|
|
INFO dropped
|
|
```
|
|
|
|
An INFO event may contain the configuration transition that caused the later error.
|
|
|
|
## Required Capabilities
|
|
|
|
C3, C4, C9.
|
|
|
|
## Expected Output
|
|
|
|
```text
|
|
Warning:
|
|
INFO and DEBUG events were excluded for 37% of the incident window.
|
|
Causal lead-up may therefore be incomplete.
|
|
```
|
|
|
|
---
|
|
|
|
# 15. UC-11 — Long-Term Regression Analysis
|
|
|
|
## Actor
|
|
|
|
Developer, SRE, analysis agent.
|
|
|
|
## Goal
|
|
|
|
Determine whether a behavior has appeared before across months, releases, or architecture changes.
|
|
|
|
## Examples
|
|
|
|
```text
|
|
"Did p99 latency change after each release?"
|
|
"Has this timeout signature appeared previously?"
|
|
"Did this failure disappear and later return?"
|
|
```
|
|
|
|
## Required Capabilities
|
|
|
|
C2, C6, C7, C11.
|
|
|
|
## Value
|
|
|
|
Long-retained evidence enables reasoning about recurring dynamics that short retention hides.
|
|
|
|
---
|
|
|
|
# 16. UC-12 — Build and Deployment Correlation
|
|
|
|
## Actor
|
|
|
|
Developer, deployment agent, incident agent.
|
|
|
|
## Goal
|
|
|
|
Determine whether an incident correlates with a particular:
|
|
|
|
- build;
|
|
- deployment;
|
|
- feature flag;
|
|
- configuration change;
|
|
- dependency version.
|
|
|
|
## Required Evidence
|
|
|
|
Observability data plus operational metadata.
|
|
|
|
## Required Capabilities
|
|
|
|
C8, C11.
|
|
|
|
## Expected Output
|
|
|
|
```text
|
|
build 2026.09.12.4
|
|
deployment started: 09:31
|
|
error increase: 09:34
|
|
affected regions: eu-central, eu-west
|
|
prior similar incident: build 2026.07.18.2
|
|
```
|
|
|
|
---
|
|
|
|
# 17. UC-13 — Rare Failure Discovery
|
|
|
|
## Actor
|
|
|
|
Agent or SRE.
|
|
|
|
## Goal
|
|
|
|
Find failures that are too rare to influence aggregates materially.
|
|
|
|
Examples:
|
|
|
|
```text
|
|
1 in 10 million corrupt response
|
|
one malformed trace
|
|
rare race condition
|
|
single customer-specific path
|
|
```
|
|
|
|
## Risk
|
|
|
|
Sampling and rollups disproportionately hide rare events.
|
|
|
|
## Required Capabilities
|
|
|
|
C1, C2, C7.
|
|
|
|
## Design Note
|
|
|
|
This is one of the strongest justifications for cheap long-term raw storage.
|
|
|
|
---
|
|
|
|
# 18. UC-14 — Trace-to-Log-to-Metric Correlation
|
|
|
|
## Actor
|
|
|
|
Incident agent.
|
|
|
|
## Goal
|
|
|
|
Move between signal types while investigating one causal chain.
|
|
|
|
Example:
|
|
|
|
```text
|
|
metric anomaly
|
|
|
|
|
v
|
|
affected service
|
|
|
|
|
v
|
|
trace cohort
|
|
|
|
|
v
|
|
specific span
|
|
|
|
|
v
|
|
related logs
|
|
|
|
|
v
|
|
deployment / host / customer context
|
|
```
|
|
|
|
## Required Capabilities
|
|
|
|
C8, C1, C7.
|
|
|
|
## Expected Output
|
|
|
|
A navigable causal evidence graph rather than isolated telemetry silos.
|
|
|
|
---
|
|
|
|
# 19. UC-15 — Agent Self-Check for Missing Evidence
|
|
|
|
## Actor
|
|
|
|
Autonomous analysis agent.
|
|
|
|
## Goal
|
|
|
|
Before committing to a diagnosis, assess whether its evidence base is complete enough.
|
|
|
|
## Required Behavior
|
|
|
|
The agent asks the platform:
|
|
|
|
```text
|
|
What relevant evidence classes exist?
|
|
Which were retained?
|
|
Which were sampled?
|
|
Which were unavailable?
|
|
Which sources failed?
|
|
```
|
|
|
|
## Required Capabilities
|
|
|
|
C4, C9, C14.
|
|
|
|
## Expected Result
|
|
|
|
Example:
|
|
|
|
```text
|
|
Reasoning confidence: 0.91
|
|
Evidence completeness: 0.38
|
|
|
|
Action:
|
|
Do not declare definitive root cause.
|
|
Search cold traces and prior incidents first.
|
|
```
|
|
|
|
---
|
|
|
|
# 20. UC-16 — Postmortem as Durable Organizational Memory
|
|
|
|
## Actor
|
|
|
|
Human team, incident agent, future analysis agent.
|
|
|
|
## Goal
|
|
|
|
Preserve causal knowledge after raw telemetry has expired.
|
|
|
|
## Persist
|
|
|
|
- incident summary;
|
|
- symptoms;
|
|
- causal chain;
|
|
- root cause;
|
|
- blast radius;
|
|
- evidence references;
|
|
- known evidence gaps;
|
|
- remediation;
|
|
- recurrence indicators;
|
|
- related incidents.
|
|
|
|
## Required Capabilities
|
|
|
|
C5, C6, C11.
|
|
|
|
## Value
|
|
|
|
The postmortem becomes a durable compressed representation of organizational learning.
|
|
|
|
---
|
|
|
|
# 21. UC-17 — Full-Fidelity Calibration Cohort
|
|
|
|
## Actor
|
|
|
|
Evaluation agent, observability platform team.
|
|
|
|
## Goal
|
|
|
|
Measure how well sampled or aggregated telemetry represents reality.
|
|
|
|
## Method
|
|
|
|
Retain selected windows at complete fidelity.
|
|
|
|
Compare:
|
|
|
|
```text
|
|
sampled query result
|
|
vs
|
|
full-fidelity query result
|
|
```
|
|
|
|
## Required Capabilities
|
|
|
|
C12, C3, C10.
|
|
|
|
## Measure
|
|
|
|
- false negatives;
|
|
- prevalence distortion;
|
|
- lost dimensions;
|
|
- root-cause accuracy;
|
|
- confidence calibration.
|
|
|
|
---
|
|
|
|
# 22. UC-18 — Telemetry Policy Impact Assessment
|
|
|
|
## Actor
|
|
|
|
Platform engineering team.
|
|
|
|
## Goal
|
|
|
|
Determine whether changing sampling, retention, or rollup policy harms investigation quality.
|
|
|
|
## Example Change
|
|
|
|
```text
|
|
normal log sampling
|
|
1% -> 0.1%
|
|
```
|
|
|
|
## Required Capabilities
|
|
|
|
C3, C10, C12.
|
|
|
|
## Compare
|
|
|
|
Before and after:
|
|
|
|
```text
|
|
storage cost
|
|
query cost
|
|
incident-answer accuracy
|
|
evidence completeness
|
|
rare-event recovery
|
|
```
|
|
|
|
## Expected Outcome
|
|
|
|
Telemetry economics become an empirical optimization problem rather than a guess.
|
|
|
|
---
|
|
|
|
# 23. UC-19 — Evidence-Preserving Cost Optimization
|
|
|
|
## Actor
|
|
|
|
Platform architect, FinOps, SRE.
|
|
|
|
## Goal
|
|
|
|
Reduce telemetry cost while preserving future investigative reach.
|
|
|
|
## Preferred Optimization Order
|
|
|
|
```text
|
|
1 compression
|
|
2 dictionary encoding
|
|
3 delta encoding
|
|
4 repeated-attribute factoring
|
|
5 columnar layout
|
|
6 cold-tier movement
|
|
7 adaptive data skipping
|
|
8 semantic compaction
|
|
9 sampling / filtering / rollups only if necessary
|
|
```
|
|
|
|
## Required Capabilities
|
|
|
|
C1, C2, C3, C7, C10.
|
|
|
|
## Design Principle
|
|
|
|
Lossless optimization should be exhausted before semantic evidence destruction becomes the primary cost lever.
|
|
|
|
---
|
|
|
|
# 24. UC-20 — Agentic Coding Failure Analysis
|
|
|
|
## Actor
|
|
|
|
Coding agent, engineering team, evaluation agent.
|
|
|
|
## Goal
|
|
|
|
Understand why an autonomous coding or deployment agent caused or failed to resolve a production problem.
|
|
|
|
## Evidence
|
|
|
|
Potentially:
|
|
|
|
- agent prompts;
|
|
- tool calls;
|
|
- deployments;
|
|
- code revisions;
|
|
- tests;
|
|
- traces;
|
|
- application logs;
|
|
- infrastructure events;
|
|
- prior similar sessions.
|
|
|
|
## Questions
|
|
|
|
```text
|
|
What did the agent believe?
|
|
What evidence was available?
|
|
What evidence was missing?
|
|
What change did it make?
|
|
What operational effect followed?
|
|
Was its confidence justified?
|
|
```
|
|
|
|
## Required Capabilities
|
|
|
|
C1, C4, C5, C8, C11, C14.
|
|
|
|
## Value
|
|
|
|
`plenty-gobble` becomes useful not only for system observability but also for evaluating agent behavior against system reality.
|
|
|
|
---
|
|
|
|
# 25. UC-21 — Security / Audit Investigation
|
|
|
|
## Actor
|
|
|
|
Security analyst or security agent.
|
|
|
|
## Goal
|
|
|
|
Reconstruct an event chain without allowing retention or filtering gaps to masquerade as exonerating evidence.
|
|
|
|
## Examples
|
|
|
|
```text
|
|
credential misuse
|
|
unexpected privilege change
|
|
suspicious access sequence
|
|
cross-tenant access attempt
|
|
```
|
|
|
|
## Required Capabilities
|
|
|
|
C1, C3, C4, C6, C8, C13.
|
|
|
|
## Special Requirement
|
|
|
|
Evidence provenance and transformation lineage should be auditable.
|
|
|
|
---
|
|
|
|
# 26. UC-22 — Customer-Specific Incident Analysis
|
|
|
|
## Actor
|
|
|
|
Support engineer, SRE, customer-facing agent.
|
|
|
|
## Goal
|
|
|
|
Determine whether and how a specific customer was affected.
|
|
|
|
## Evidence Dimensions
|
|
|
|
```text
|
|
tenant
|
|
customer
|
|
account
|
|
region
|
|
service
|
|
build
|
|
request path
|
|
```
|
|
|
|
## Risk
|
|
|
|
Customer or account dimensions are often high-cardinality and therefore candidates for removal.
|
|
|
|
## Required Capabilities
|
|
|
|
C1, C4, C8, C13.
|
|
|
|
## Expected Output
|
|
|
|
```text
|
|
confirmed affected requests
|
|
time window
|
|
affected features
|
|
related incidents
|
|
evidence completeness
|
|
```
|
|
|
|
---
|
|
|
|
# 27. UC-23 — Regional Incident Comparison
|
|
|
|
## Actor
|
|
|
|
Operator or agent.
|
|
|
|
## Goal
|
|
|
|
Compare one region against another during an incident.
|
|
|
|
## Example
|
|
|
|
```text
|
|
eu-central-1 fails
|
|
eu-west-1 does not
|
|
```
|
|
|
|
Then ask:
|
|
|
|
```text
|
|
What differs?
|
|
build?
|
|
configuration?
|
|
dependency endpoint?
|
|
traffic shape?
|
|
customer mix?
|
|
```
|
|
|
|
## Required Capabilities
|
|
|
|
C1, C4, C8.
|
|
|
|
## Risk
|
|
|
|
Rollups may preserve the region but remove the dimension explaining the difference.
|
|
|
|
---
|
|
|
|
# 28. UC-24 — Incident Case Handoff
|
|
|
|
## Actor
|
|
|
|
Human-to-agent, agent-to-human, agent-to-agent.
|
|
|
|
## Goal
|
|
|
|
Transfer incident context without silently losing evidence boundaries.
|
|
|
|
## Package
|
|
|
|
```text
|
|
Case File
|
|
+
|
|
Evidence Envelope
|
|
+
|
|
Loss Manifests
|
|
+
|
|
query provenance
|
|
+
|
|
open questions
|
|
```
|
|
|
|
## Required Capabilities
|
|
|
|
C5, C4, C10.
|
|
|
|
## Design Principle
|
|
|
|
A handoff should transfer both:
|
|
|
|
```text
|
|
what we know
|
|
```
|
|
|
|
and:
|
|
|
|
```text
|
|
why we believe the available evidence is sufficient or insufficient
|
|
```
|
|
|
|
---
|
|
|
|
# 29. UC-25 — Confidence-Gated Automated Remediation
|
|
|
|
## Actor
|
|
|
|
Supervisory agent or autonomous remediation system.
|
|
|
|
## Goal
|
|
|
|
Automatically take corrective action only when evidence quality justifies it.
|
|
|
|
## Example Policy
|
|
|
|
```text
|
|
if root_cause_confidence >= 0.9
|
|
and evidence_completeness >= 0.8
|
|
and blast_radius_confidence >= 0.8
|
|
then remediation may execute automatically
|
|
|
|
otherwise require human approval
|
|
```
|
|
|
|
## Required Capabilities
|
|
|
|
C4, C9, C14.
|
|
|
|
## Value
|
|
|
|
This turns completeness metadata into an operational safety control.
|
|
|
|
---
|
|
|
|
# 30. Secondary Use Cases
|
|
|
|
The architecture also naturally supports several adjacent scenarios.
|
|
|
|
## UC-S1 — Capacity Planning
|
|
|
|
Use long-term full-fidelity traffic history to understand workload shape rather than only rolled-up averages.
|
|
|
|
## UC-S2 — Performance Regression Search
|
|
|
|
Compare latency distributions across versions and deployments.
|
|
|
|
## UC-S3 — Reliability Benchmarking
|
|
|
|
Evaluate whether operational reliability changes after architecture or infrastructure modifications.
|
|
|
|
## UC-S4 — SLA / SLO Evidence
|
|
|
|
Provide auditable underlying evidence for calculated service indicators.
|
|
|
|
## UC-S5 — Agent Evaluation Dataset Generation
|
|
|
|
Construct controlled complete and degraded evidence packages for evaluating analysis-agent robustness.
|
|
|
|
## UC-S6 — Observability Policy Simulation
|
|
|
|
Estimate the effect of a proposed sampling or retention policy before applying it in production.
|
|
|
|
---
|
|
|
|
# 31. Use Cases by Evidence Fidelity Requirement
|
|
|
|
| Use Case | Minimum Useful Fidelity |
|
|
|---|---|
|
|
| Unknown-cause investigation | Near-lossless / lossless preferred |
|
|
| Exact security investigation | Lossless strongly preferred |
|
|
| Rare failure discovery | Lossless |
|
|
| Trend analysis | Aggregated may suffice |
|
|
| Capacity planning | Aggregated / near-lossless |
|
|
| Customer blast radius | Raw dimensions required |
|
|
| Historical precedent | Case File may suffice |
|
|
| Deployment comparison | Near-lossless |
|
|
| Agent evaluation | Controlled fidelity levels required |
|
|
| Automated remediation | High evidence completeness required |
|
|
|
|
This demonstrates that `plenty-gobble` should not impose one universal retention strategy.
|
|
|
|
It should make fidelity an explicit, queryable property.
|
|
|
|
---
|
|
|
|
# 32. Use Cases by Latency Requirement
|
|
|
|
| Class | Example | Typical Latency Preference |
|
|
|---|---|---:|
|
|
| Immediate | active incident hot logs | sub-second to seconds |
|
|
| Interactive | trace correlation | seconds |
|
|
| Investigative | 30-day historical search | seconds to tens of seconds |
|
|
| Archival | prior-year precedent | tens of seconds or more acceptable |
|
|
| Evaluation | agent benchmark corpus | batch-oriented |
|
|
|
|
This supports the principle:
|
|
|
|
> Older data may become slower without becoming unavailable.
|
|
|
|
---
|
|
|
|
# 33. Use Cases Driving the Storage Model
|
|
|
|
The catalog implies several storage requirements.
|
|
|
|
## Requirement 1 — Keep raw semantics longer than hot performance
|
|
|
|
The system should distinguish:
|
|
|
|
```text
|
|
availability
|
|
```
|
|
|
|
from:
|
|
|
|
```text
|
|
latency
|
|
```
|
|
|
|
## Requirement 2 — Loss transformations are first-class records
|
|
|
|
Sampling and rollup are not implementation trivia.
|
|
|
|
## Requirement 3 — The incident archive has a different retention horizon
|
|
|
|
Postmortems and Case Files may outlive raw telemetry by years.
|
|
|
|
## Requirement 4 — Query-time pruning should be aggressive
|
|
|
|
Physical I/O can be minimized without discarding future evidence.
|
|
|
|
## Requirement 5 — High-cardinality dimensions may be epistemically important
|
|
|
|
Storage policy should not equate expensive dimensions with unimportant dimensions.
|
|
|
|
---
|
|
|
|
# 34. Use Cases Driving the Query Model
|
|
|
|
The query model should eventually support:
|
|
|
|
```text
|
|
telemetry query
|
|
+
|
|
coverage query
|
|
+
|
|
precedent query
|
|
+
|
|
incident query
|
|
```
|
|
|
|
A future agent-facing request might effectively be:
|
|
|
|
```text
|
|
Find traces matching this symptom,
|
|
show related logs and metric anomalies,
|
|
find prior incidents,
|
|
and tell me what telemetry was unavailable.
|
|
```
|
|
|
|
This is richer than conventional analytical SQL alone.
|
|
|
|
---
|
|
|
|
# 35. Use Cases Driving the Case File Model
|
|
|
|
The Case File should likely become a first-class `plenty-gobble` domain object.
|
|
|
|
Candidate structure:
|
|
|
|
```text
|
|
CaseFile
|
|
id
|
|
tenant_id
|
|
incident_id
|
|
|
|
observation
|
|
title
|
|
comments
|
|
|
|
evidence_sets[]
|
|
logs
|
|
traces
|
|
metrics
|
|
events
|
|
external context
|
|
|
|
precedent[]
|
|
completeness
|
|
loss_manifests[]
|
|
|
|
findings
|
|
root_cause
|
|
evidence
|
|
precedent
|
|
blast_radius
|
|
|
|
confidence
|
|
reasoning
|
|
completeness
|
|
reliability
|
|
|
|
open_questions[]
|
|
```
|
|
|
|
---
|
|
|
|
# 36. Use Cases Driving Agent Interfaces
|
|
|
|
Agent-oriented interfaces should make it easy to ask:
|
|
|
|
```text
|
|
What evidence exists?
|
|
|
|
What evidence should exist?
|
|
|
|
What evidence is missing?
|
|
|
|
What was transformed?
|
|
|
|
What was retained at full fidelity?
|
|
|
|
What prior incidents resemble this?
|
|
|
|
What conclusion is supported?
|
|
|
|
What conclusion is not justified?
|
|
```
|
|
|
|
The completeness surface is therefore as important as the data-query surface.
|
|
|
|
---
|
|
|
|
# 37. Suggested MVP Use-Case Set
|
|
|
|
The first implementation should not attempt the entire catalog.
|
|
|
|
A focused MVP should support five representative cases:
|
|
|
|
## MVP-1 — Selective historical lookup
|
|
|
|
Find one trace ID across hot and cold telemetry.
|
|
|
|
Validates:
|
|
|
|
- segment catalog;
|
|
- pruning;
|
|
- hot/cold query.
|
|
|
|
## MVP-2 — Evidence completeness query
|
|
|
|
Report retention, sampling, and known gaps for a time range.
|
|
|
|
Validates:
|
|
|
|
- Loss Manifest;
|
|
- completeness model.
|
|
|
|
## MVP-3 — Case File construction
|
|
|
|
Bundle logs, traces, metrics, and completeness metadata.
|
|
|
|
Validates:
|
|
|
|
- incident evidence model.
|
|
|
|
## MVP-4 — Prior incident retrieval
|
|
|
|
Retrieve one structurally similar historical incident.
|
|
|
|
Validates:
|
|
|
|
- durable incident archive.
|
|
|
|
## MVP-5 — Agent confidence gating
|
|
|
|
Demonstrate that incomplete evidence changes the permitted conclusion or action.
|
|
|
|
Validates:
|
|
|
|
- agentic value proposition.
|
|
|
|
---
|
|
|
|
# 38. Evaluation Scenarios
|
|
|
|
The catalog should eventually be backed by reproducible scenarios.
|
|
|
|
## Scenario A — Complete Evidence
|
|
|
|
Agent receives full telemetry.
|
|
|
|
Expected:
|
|
|
|
```text
|
|
high root-cause accuracy
|
|
appropriate confidence
|
|
```
|
|
|
|
## Scenario B — Silent Sampling
|
|
|
|
Agent receives 10% sampling but no metadata.
|
|
|
|
Expected failure mode:
|
|
|
|
```text
|
|
false certainty
|
|
```
|
|
|
|
## Scenario C — Explicit Sampling
|
|
|
|
Same data plus Loss Manifest.
|
|
|
|
Expected improvement:
|
|
|
|
```text
|
|
better confidence calibration
|
|
more requests for additional evidence
|
|
```
|
|
|
|
## Scenario D — Rollup Removes Customer Dimension
|
|
|
|
Agent asked for customer blast radius.
|
|
|
|
Expected:
|
|
|
|
```text
|
|
correct refusal to infer unavailable customer scope
|
|
```
|
|
|
|
## Scenario E — Cold Historical Precedent
|
|
|
|
Old raw data is slower but still accessible.
|
|
|
|
Expected:
|
|
|
|
```text
|
|
agent waits and improves diagnosis
|
|
```
|
|
|
|
## Scenario F — Raw Data Expired, Case File Survives
|
|
|
|
Expected:
|
|
|
|
```text
|
|
historical precedent remains useful
|
|
but evidence limitations are explicit
|
|
```
|
|
|
|
---
|
|
|
|
# 39. Prioritization
|
|
|
|
A suggested priority model:
|
|
|
|
## P0 — Foundational
|
|
|
|
- UC-01 Unknown-Cause Incident Investigation
|
|
- UC-02 Evidence Completeness Check
|
|
- UC-05 Root-Cause Evidence Package
|
|
- UC-06 Cold-Tier Historical Investigation
|
|
- UC-15 Agent Self-Check
|
|
|
|
## P1 — High Value
|
|
|
|
- UC-03 Historical Precedent Search
|
|
- UC-04 Blast-Radius Determination
|
|
- UC-07 Sampled-Telemetry Interpretation
|
|
- UC-14 Cross-Signal Correlation
|
|
- UC-16 Durable Postmortem Memory
|
|
- UC-25 Confidence-Gated Remediation
|
|
|
|
## P2 — Expansion
|
|
|
|
- customer-specific analysis;
|
|
- regional comparison;
|
|
- security investigations;
|
|
- telemetry-policy impact assessment;
|
|
- coding-agent evaluation.
|
|
|
|
---
|
|
|
|
# 40. Catalog Summary
|
|
|
|
The central use cases reveal that `plenty-gobble` is not merely a cheaper observability store.
|
|
|
|
Its stronger role is:
|
|
|
|
> **an evidence infrastructure for agents that preserves investigative reach, makes information loss explicit, and supplies enough context for agents to know when they do not know.**
|
|
|
|
The storage engine enables this through:
|
|
|
|
```text
|
|
lossless compression
|
|
columnar storage
|
|
cheap cold retention
|
|
adaptive pruning
|
|
```
|
|
|
|
The epistemic layer enables it through:
|
|
|
|
```text
|
|
Loss Manifests
|
|
Evidence Completeness
|
|
Case Files
|
|
Incident Memory
|
|
Evidence-Aware Confidence
|
|
```
|
|
|
|
Together these capabilities support a form of observability designed not only to answer today's dashboards, but to survive tomorrow's unknown questions.
|