Meta Analysis

Meta-Analysis

A pooled number is powerful and dangerous. Done on comparable studies it sharpens the truth; done on studies that measured different things it manufactures a precise-looking lie. The judgment this skill enforces is not "how do I average these" — it is "should these be averaged at all?"

When to use

Whenever the agent is tempted to report one effect size (a risk ratio, an odds ratio, a mean difference, an HR) drawn from more than one study.

The rule

Only pool comparable primary studies. When they are not comparable, show them individually and say why.

Before combining, every study in the pool must share:

1. Population — same clinical group, not "adults" vs "elderly with comorbidity" flattened together.
2. Intervention / exposure — same drug, comparable dose and route; same exposure definition.
3. Comparator — placebo vs active control are different questions; do not mix them into one pool.
4. Outcome and its measure — the same endpoint, defined and measured the same way, on the same scale.
5. Design tier — do not pool RCTs with observational studies into a single estimate. Report them in separate strata.

If any of these differ materially, do not pool. Plot each study's own estimate with its confidence interval (a forest-style layout) and let the reader see the spread.

Doing the pool responsibly

• Weight by precision, not by vote. Larger, tighter studies carry more weight; do not average raw numbers as if each study counted once.
• Report heterogeneity. State how much the studies disagree (e.g. an I² and the between-study variance). High heterogeneity is a reason to stop pooling, not a footnote.
• Keep confidence intervals bound to their estimate. A CI that has drifted from the number it belongs to is a parsing bug that becomes a clinical error. Verify each estimate carries its own interval.
• Prefer primary studies. Do not pool a review's summary figure together with the primary trials it already summarizes — that double-counts.

When you cannot pool

Say it plainly and show the evidence anyway:

These 6 studies cannot be pooled: 4 measured 28-day mortality, 2 measured
90-day; comparators differ (placebo vs standard care). Individual estimates:

  Study A (RCT, placebo, 28d)   RR 0.81  [0.70–0.94]
  Study B (RCT, placebo, 28d)   RR 0.88  [0.79–0.98]
  Study C (obs, standard, 90d)  RR 0.72  [0.55–0.95]
  ...

A row of honest individual estimates is more useful than one dishonest summary. The skill is knowing the difference.

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: Hash-7777
• Source: Hash-7777/Hash-Medical-Reasearch-Agent-Skills
• License: MIT

Install and usage instructions live in the source repository linked above.