Data Monitoring

Data Monitoring

Track metrics that matter for product success.

What This Skill Does

Sets up and monitors product metrics including core KPIs, user behavior metrics, and business indicators. Detects anomalies, tracks trends, and generates alerts. The output provides the data foundation for impact analysis and iteration planning.

When to Use

Activate this skill when:

• User wants to track product metrics or KPIs
• Phrases like "monitor X", "track Y metric", "how is it performing"
• Setting up dashboards or alerting
• Analyzing trends or detecting anomalies
• User says "set up monitoring" or "create dashboard"

How It Works

The monitoring process ensures comprehensive metric tracking:

1. Define metrics - Identify what to measure based on product goals
2. Set up tracking - Configure data collection and storage
3. Establish baselines - Determine normal ranges and thresholds
4. Configure alerts - Set up anomaly detection and notifications
5. Monitor trends - Track changes over time
6. Generate insights - Identify patterns and issues

Input Parameters

| Parameter | Type | Required | Description | |:---|:---|:---|:---| | metric_type | string | Yes | Type of metric (e.g., "engagement", "revenue", "technical") | | kpi_name | string | Yes | Specific KPI to monitor | | threshold | object | No | Alert thresholds (e.g., {"min": 100, "max": 1000}) | | frequency | string | No | Check frequency (default: "daily") |

Metric Categories

| Category | Example Metrics | Use For | |:---|:---|:---| | engagement | DAU, WAU, MAU, session length | User engagement | | acquisition | Signups, conversion rate, CAC | Growth tracking | | retention | Day-7, Day-30, churn rate | User retention | | revenue | MRR, ARPU, LTV | Business health | | technical | Error rate, latency, uptime | System health | | feature | Feature adoption, usage frequency | Feature success |

Output Structure

The skill generates monitoring outputs:

1. Alert notifications - Real-time alerts when thresholds breached
2. Trend reports - Regular summaries of metric performance
3. Dashboard specifications - Dashboard configuration recommendations

Alert Format

{
  "alert": {
    "id": "alert-uuid",
    "timestamp": "2026-03-12T10:30:00Z",
    "severity": "warning",
    "metric": "daily_active_users",
    "current_value": 850,
    "threshold": {"min": 1000},
    "change": "-15%",
    "message": "DAU dropped below threshold",
    "suggested_actions": [
      { "action": "Check if deployment caused issues", "confidence": "low", "basis": "Correlation with recent deployment" }
    ]
  }
}

Trend Report Format

{
  "report": {
    "period": "2026-03-06 to 2026-03-12",
    "metrics": [
      {
        "name": "daily_active_users",
        "current": 1050,
        "previous": 1000,
        "change": "+5%",
        "trend": "up",
        "status": "healthy",
        "data_source": "internal analytics",
        "benchmark": { "value": "N/A", "confidence": "unknown", "note": "No industry benchmark found for this metric" }
      },
      {
        "name": "conversion_rate",
        "current": "3.2%",
        "previous": "3.5%",
        "change": "-8.6%",
        "trend": "down",
        "status": "warning"
      }
    ],
    "anomalies": [
      {
        "metric": "conversion_rate",
        "detected_at": "2026-03-10",
        "description": "Unusual drop after deployment"
      }
    ],
    "recommendations": [
      "Investigate conversion rate drop",
      "Consider A/B testing checkout flow"
    ]
  }
}

Anomaly Detection

| Type | Description | Detection Method | |:---|:---|:---| | Spike | Sudden increase | Value > 3σ from mean | | Drop | Sudden decrease | Value < 3σ from mean | | Trend | Gradual change | Linear regression slope | | Pattern | Repeating pattern | Time series analysis |

Anti-Hallucination Rules (Self-Contained)

The following rules are mandatory for this skill. They are inlined here for standalone installation compatibility.

1. Mandatory Search First

引用行业基准数据前必须先搜索验证。

• 禁止仅凭训练记忆输出任何具体数据（基准值、行业对比等）
• 每个数据点必须来自实际搜索和访问的页面
• 搜索查询应包含年份，例如 SaaS conversion rate benchmark 2025 2026

2. Every Claim Must Have a Source

每个指标基准值、行业对比数据必须有来源。

• 没有来源 = 不得写入
• 来源格式：{ "claim": "...", "source": "https://...", "source_name": "...", "fetched_at": "..." }

3. Unknown is Acceptable

无法获取的真实指标标注 "Unknown"，禁止编造基线数据。标注未知是诚实，不是失败。

4. Confidence Rating

关键基准数据标注 confidence (high/medium/low)。

• high: 权威基准报告、官方统计
• medium: 可靠行业报告、公开基准数据
• low: 社区讨论、博客、推测性分析

5. Distinguish Fact from Inference

区分实际监控数据和推断的异常原因。推断必须标注 basis 和 confidence。

6. Quality Gate (Non-negotiable)

以下任一条未满足，不得输出结果：

• [ ] 所有具体基准/对比声明都有来源 URL
• [ ] 无法验证的数据已标注 "Unknown"
• [ ] 每个数据点标注了 confidence 等级
• [ ] facts 和 inferences 已明确区分
• [ ] 搜索记录已附在输出中

Search Record

每个分析输出必须附带搜索记录：{ "search_queries_used": [...], "sources_accessed": [{ "url": "...", "title": "...", "used_for": "..." }] }

Quality Standards

Before delivering, the monitoring setup should:

• Define clear metrics with business relevance
• Set appropriate thresholds based on real historical data or industry benchmarks
• ALL industry benchmarks cited have source URLs
• ALL suggested root causes have confidence rating
• Facts (actual metric values) and inferences (suspected causes) are clearly distinguished
• Include actionable alert messages
• Provide context for metric interpretation
• Specify data sources and collection methods

Context Integration

Reads:

• docs/product/prd/*.md - Success metrics defined in PRDs

Writes:

• docs/product/.ompm/metrics.json - Current metric values and trends

Read By:

• impact-analysis - Uses metric data for post-release analysis
• iteration-planning - References trends for planning decisions

Example Usage

User: "Monitor our daily active users"
→ Sets up DAU monitoring with alerts

User: "Create a dashboard for key metrics"
→ Generates dashboard specification with core KPIs

User: "Why did conversion rate drop yesterday?"
→ Analyzes anomaly, provides root cause suggestions

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: kelegele
• Source: kelegele/oh-my-pm
• License: MIT

Install and usage instructions live in the source repository linked above.