Crabpot Perf Metrics

Purpose

Turn Crabpot/OpenClaw perf artifacts into a clear read: what changed, what is real signal, what is stale/noisy, and what action is worth taking.

When to use

• Comparing openclaw/crabpot branches such as main, crab-beta, or crab-development.
• Reading reports/crabpot-dashboard-data.json, reports/crabpot-import-loop-profile.json, reports/crabpot-runtime-profile.json, README dashboard metrics, or GitHub Actions report artifacts.
• The user asks about perf, runtime, CPU, RSS, import loop, OpenClaw import/activate, dashboard deltas, or whether a Crabpot performance change is meaningful.

Inputs

• Target repo or local checkout, usually openclaw/crabpot.
• Branches, refs, PRs, or artifact URLs to compare.
• Optional target OpenClaw labels/SHAs if the comparison crosses openclaw@latest, beta, or openclaw/openclaw@main.

Workflow

1. Establish freshness before interpreting numbers.

• Resolve branch heads and report timestamps.
• Compare dashboard JSON with raw profile JSON on the same branch.
• If one branch is stale, say so first and avoid strong conclusions.
• Prefer raw JSON over README snippets when they disagree.

1. Separate the metric families.

• import-loop-profile: cheap cold capture loop against a tiny fixture. Good for harness/import drift, weak for production runtime claims.
• runtime-profile: macro command timings across Crabpot analysis commands. Better trend signal for real suite cost.
• OpenClaw lifecycle: import plus activate phases. If openClawLifecycleCount is 0, the branch dashboard does not include this signal.

1. Interpret import-loop metrics conservatively.

• Use p50WallMs, p95WallMs, p50PluginWallDeltaMs, p95PluginWallDeltaMs, maxPluginPeakRssDeltaMb, and maxPluginCpuDeltaMsEstimate.
• Treat 3-run p95 as "worst of three", not a stable tail latency estimate.
• A small wall-time bump with flat plugin RSS/CPU usually means jitter or module-resolution overhead, not a memory/CPU regression.
• Baseline-adjusted plugin deltas are more useful than raw wall/RSS/CPU values.

1. Interpret runtime profile metrics as the stronger trend.

• Use summary.p50WallMs, summary.p95WallMs, summary.maxPeakRssMb, summary.maxCpuMsEstimate, and summary.maxHarnessHeapDeltaMb.
• Inspect per-command medians for the source of change: fixture-inspection, compat-report-registry, contract-capture, synthetic-probe-plan, cold-import-readiness, workspace-plan, platform-probes, and import-loop-profile.
• Broadly uniform slowdowns across target-aware commands usually point at target OpenClaw surface parsing or registry work, not one plugin fixture.
• If RSS is flat and CPU moves only slightly, call it a wall-clock/runtime overhead change, not a memory leak.

1. Account for surface changes.

• Different fixture counts, entrypoint counts, OpenClaw labels, or commit SHAs make the comparison partly apples-to-oranges.
• New fixtures can raise issue/probe counts while leaving perf healthy.
• OpenClaw main vs npm latest can improve compatibility while adding modest analysis overhead.

1. Give the user a direct read.

• Start with the conclusion.
• Prefer bullets and a short summary over tables unless the user asks for a table.
• Include exact deltas and classify them: real regression, modest slowdown, noise, stale data, or missing signal.
• Recommend one next action: refresh dashboard, add a dedicated lifecycle lane, investigate a specific command, or ignore as noise.

Useful Commands

Use gh if ghx is unavailable.

Fetch current branch artifacts from GitHub:

gh api /repos/openclaw/crabpot/contents/reports/crabpot-dashboard-data.json?ref=crab-development

Prefer a small Node or jq extraction over manual README scraping. Pull these files when available:

• reports/crabpot-dashboard-data.json
• reports/crabpot-import-loop-profile.json
• reports/crabpot-runtime-profile.json
• reports/crabpot-ci-summary.json

Outputs

• Concise perf summary in bullets.
• Freshness caveats when needed.
• Important deltas with interpretation.
• Clear owner/action recommendation, especially when lifecycle import+activate is missing from dashboard artifacts.

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: vincentkoc
• Source: vincentkoc/dotskills
• License: MIT

Install and usage instructions live in the source repository linked above.