Pilot Analysis
Use this skill to compute and inspect ACP metrics, telemetry, and export outputs for intervention and baseline cycles without overclaiming.
Install
$ agentstack add skill-jasontang-ai-acp-pilot-analysis ✓ scanned · ✓ verified, works with Claude Code, Cursor, and more.
Security review
✓ PassedNo issues found. Passed automated security review. · v0.1.0 How review works →
- ✓ Prompt-injection patterns
- ✓ Secret / credential exfiltration
- ✓ Dangerous shell & filesystem operations
- ✓ Untrusted network calls
- ✓ Known-malicious package signatures
What it can access
- ✓ Network access No
- ✓ Filesystem access No
- ✓ Shell / process execution No
- ✓ Environment & secrets No
- ✓ Dynamic code execution No
From automated source analysis of v0.1.0. “Used” means the capability is present in the source — more access means more to trust, not that it’s unsafe.
Verified badge
Passed review? Show it. Paste this badge into your README, it links to the public security report.
Links to your public security report.[](https://agentstack.voostack.com/security/report/skill-jasontang-ai-acp-pilot-analysis) Reliability & compatibility
Declared compatibility
Compatibility is declared by the source manifest. End-to-end runtime verification is coming, see below.
We're building live execution health for every listing: tool-call success rate, median latency, uptime, and last-checked timestamps, measured, not self-reported. It isn't live yet, so we don't show numbers we can't stand behind.
How agent discovery & health will work →About
What this skill is for
Use this skill when a pilot, benchmark, or rehearsal needs condition-comparable analysis rather than raw cycle inspection.
When to use it
- Summarizing one or more cycles.
- Comparing intervention and baseline conditions.
- Preparing pilot evidence for operator or reviewer handoff.
When not to use it
- Do not use it to manage lifecycle transitions.
- Do not use it to author participant content.
- Do not convert prototype metrics into field-efficacy claims.
Inputs expected
- One or more cycle ids or cycle JSON files
- Condition labels
- Optional export mode
- Claim boundary for the analysis
Preflight / prerequisites
- Confirm cycles are comparable.
- Confirm exports and metrics are generated by Relay.
- Separate benchmark evidence, pilot evidence, and field-efficacy claims.
Workflow
- Run one-cycle metric summaries.
- Run condition comparison when at least two cycles exist.
- Inspect missing metrics and telemetry gaps.
- Summarize only what the evidence supports.
- Call out non-claims explicitly.
Decision rules / judgment criteria
- Comparable denominators matter more than impressive numbers.
- Missing telemetry is a gap.
- Synthetic benchmark evidence supports operational feasibility, not field efficacy.
- Baseline and intervention condition counts should remain aligned where the scenario requires it.
Escalation rules
Escalate when intervention and baseline differ in participant or contribution count unexpectedly, when key metrics are missing, or when a requested claim exceeds the evidence class.
Available scripts
scripts/metrics-summary.mjsscripts/compare-cycles.mjs [...]
Outputs
- Metrics summary
- Condition comparison
- Explicit evidence gaps
- Claim-safe analysis notes
Failure handling
If telemetry is incomplete, report incompleteness and stop short of numeric inference.
Trust / safety notes
Pilot analysis is vulnerable to overclaiming. Treat every output as a bounded evidence artifact.
Composition notes
Use inside pilot-cycle-review and foresight-submission-prep.
Examples to inspect next
Read examples/casebook.md for condition comparison and overclaiming anti-patterns.
Evaluation hooks
Run scripts/metrics-summary.mjs, scripts/compare-cycles.mjs, and npm run skills:audit.
Source & license
This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.
- Author: jasontang-ai
- Source: jasontang-ai/acp
- License: Apache-2.0
Install and usage instructions live in the source repository linked above.
Reviews
No reviews yet, be the first.
Write a review
Versions
- v0.1.0 Imported from the upstream source.