Learning meta-analysis: the statistics you need first
Meta-analysis is the statistical machinery behind every serious systematic review — the tool that pulls the signal out of a stack of studies that individually might mean anything. It's how evidence-based medicine builds conclusions and how newer fields (ML fairness, education intervention research) are learning to consolidate their own literature. This is a staged path through the on-ramp, with an honest note on which parts a general-purpose stats platform helps with.
What meta-analysis actually is
A meta-analysis is a statistical synthesis of multiple studies that address the same question. It converts each study's result into a common effect-size metric with a variance, weights the studies (typically by inverse variance), and pools them into a single summary estimate with a confidence interval. It also quantifies how much the studies disagree with each other (heterogeneity), and offers a set of tools to probe why.
Done well, a meta-analysis is the most authoritative form of evidence in most fields. Done badly, it can launder a stack of biased or heterogeneous studies into a single confident-sounding number. The methodology — protocol registration, transparent search, risk-of-bias assessment, pre-specified analysis — is what separates the two.
A meta-analysis is only as good as the studies it pools and the transparency with which it pools them. The number in bold at the top of the forest plot is the last thing you should look at.
A staged learning path
Meta-analysis teaches best in three layers: the effect-size and inference fluency underneath, the pooling methodology itself, and the specialist extensions (network meta-analysis, IPD, Bayesian).
Layer 1 — statistical foundations
The bedrock:
- Effect sizes — the atomic unit of meta-analysis. Cohen's d and Hedges' g for continuous outcomes; log-OR / log-RR / log-HR for binary and time-to-event; Pearson r for correlational data. Know what each one measures and how it's computed from the primary study's numbers.
- Confidence intervals — meta-analytic pooling weights each study by the inverse of its variance, so understanding what a CI is (and what makes it narrow or wide) is directly load-bearing.
- Weighted means — because the pooled estimate is a weighted average of the study estimates, weighted by precision. The arithmetic that every fixed-effect meta-analysis reduces to.
- P-value fluency — enough to read primary studies critically. Meta-analysis is estimation-first, but you still need to know what the primary studies were testing.
- Reading a regression table — many primary studies report an adjusted regression coefficient rather than a raw group difference. Extracting the right number for the pool requires comfort reading these tables.
Layer 2 — meta-analytic methodology
The synthesis layer:
- Fixed-effect vs random-effects models — the single most important modelling choice. Fixed-effect assumes one true effect; random-effects assumes a distribution of true effects. Random-effects is the modern default for almost every real-world meta.
- Heterogeneity — Cochran's Q (test), I² (percentage of variance from heterogeneity, not sampling error), τ² (the actual variance component). Reporting all three is standard.
- Forest plots — the canonical visualisation. Each row is a study; the diamond at the bottom is the pooled estimate. Fluency in reading them is the fastest way to build meta-analytic intuition.
- Publication bias — funnel plots, Egger's regression test, trim-and-fill, selection models. The tools for detecting whether the pooled estimate is inflated because null findings never made it to publication.
- Subgroup analysis and meta-regression — the tools for asking "does the effect depend on study characteristics?" (patient age, follow-up length, intervention intensity). Powerful when pre-specified; treacherous when fished for.
- Sensitivity analysis — leave-one-out, alternative effect-size metrics, alternative weighting schemes. The check that the conclusion isn't being driven by a single outlier study.
- PRISMA 2020 — the reporting checklist. Reading it once tells you what a well-reported systematic review looks like.
Layer 3 — specialist extensions and tools
Once the standard pooling is fluent:
- Network meta-analysis (NMA) — combines direct and indirect comparisons across a network of interventions, so you can rank multiple treatments even when no head-to-head trial exists. Widely used in HTA (health-technology assessment).
- Individual patient data (IPD) meta-analysis — instead of pooling summary statistics from each study, you pool the raw participant-level data. Much more powerful; much harder to obtain.
- Bayesian meta-analysis — natural for random-effects models, and increasingly the standard for NMA (via WinBUGS or Stan).
- Dose-response meta-analysis — for exposure-response relationships (e.g. dietary recommendations from cohort data).
- Diagnostic-accuracy meta-analysis — bivariate models for pooling sensitivity and specificity jointly.
The tools working meta-analysts actually use:
- metafor (R) — Viechtbauer's package; the reference implementation. Comprehensive, well-documented, actively maintained.
- meta / netmeta (R) — friendlier syntax than metafor; netmeta is the standard for frequentist network meta-analysis.
- robvis (R) — for generating the risk-of-bias visualisations that go alongside the forest plot.
- RevMan — the Cochrane Collaboration's free tool; the standard for Cochrane reviews specifically. GUI-driven, approachable for beginners.
- Comprehensive Meta-Analysis (CMA) — commercial GUI tool; common in social sciences.
Where thericerca can help — and where it can't
Meta-analysis is the field on this list where thericerca does the least of the actual work — the pooling itself lives in dedicated tools like metafor and RevMan. What thericerca does deliver is a big chunk of the fluency you need to read a meta-analysis critically and to compute effect sizes from primary studies (either your own, or ones you're extracting for a review).
Fits thericerca well. Computing effect sizes and their CIs for individual studies (Cohen's d from t-tests, r from correlations, Cramér's V from chi-square, HR from Cox regression). Reproducing a primary study's analysis before pooling it (a required sanity-check for any careful reviewer). Running the underlying t-tests, ANOVAs, chi-squares, and regressions whose effect sizes you'll extract. Understanding the confidence intervals whose reciprocals become the meta-analytic weights. The reproducibility export gives you a Jupyter notebook of the individual-study analysis, ready to be handed over as an evidence trail.
Outside thericerca's scope. The pooled estimate itself. Random-effects models with τ² and I². Forest plots. Funnel plots and publication-bias diagnostics (Egger's, trim-and-fill, selection models). Subgroup and meta-regression tests. Network meta-analysis. Individual patient data meta-analysis. Bayesian meta-analysis. Diagnostic-accuracy meta-analysis. For all of those, learn R (metafor, meta, netmeta) or RevMan.
Resources worth your time
- Books. Borenstein, Hedges, Higgins, Rothstein, Introduction to Meta-Analysis, is the standard first text. The Cochrane Handbook for Systematic Reviews of Interventions is free online and definitive on methodology. Harrer, Cuijpers, Furukawa, Ebert, Doing Meta-Analysis with R is a free bookdown that pairs directly with the metafor / meta packages.
- Tutorials. The metafor package's own website (metafor-project.org) has extensive worked examples for every model type. Cochrane's Training portal has free short courses aligned with the Handbook.
- Reporting. The PRISMA 2020 statement and its checklist are what every reputable journal expects for systematic reviews. Reading the checklist once teaches you what a well-reported review looks like.
- Practice datasets. The metafor package bundles dozens of real published meta-analyses (BCG vaccine, dietary sodium, educator praise, and many more) with the raw effect sizes and study characteristics — ideal for learning by re-running.
- Community. The Society for Research Synthesis Methodology (SRSM), the Campbell Collaboration (for social-science reviews), and the r-sig-meta-analysis mailing list are where working practitioners solve problems in public.
A concrete first month
If you're starting with a working stats background, spend Week 1 on effect sizes — until you can convert a t-statistic, an F-statistic, a 2×2 table, and a Pearson r into a Cohen's d (or a log-OR / log-RR) in your head. Week 2, work through fixed- and random-effects models on a bundled metafor dataset. Week 3, heterogeneity assessment and publication-bias diagnostics. Week 4, do a mock mini-review: pull five real published studies on one narrow topic, extract the effect sizes, pool them, and write up the result.
thericerca can carry a real slice of that work — computing the effect sizes you'll pool, reproducing individual studies' analyses to sanity-check the numbers before you enter them into metafor, and giving you the effect-size fluency that makes the pooling read intuitively. Use the guided study pipeline to reproduce individual studies from their raw data (when available), and use the test-picker to double-check that a paper's chosen test matches what you'd extract from.
For the pooling itself and the specialist extensions — NMA, IPD, Bayesian — you'll graduate to R (metafor / meta / netmeta) or RevMan. That's the way in, and there's no shortcut. But the effect-size and confidence-interval fluency you build first is what keeps the forest plot from looking like a random pile of confidence intervals when you get there.
The short version. Learn statistics and effect sizes fluently, then meta-analytic methodology, then the specialist extensions — in that order. thericerca doesn't do meta-analysis itself, but it gets you fluent in the individual-study analyses whose effect sizes you'll pool. When someone shows you a meta-analysis, look at the forest-plot heterogeneity, the funnel plot, and the risk-of-bias summary before the pooled diamond. If they haven't reported all three, the diamond isn't the whole story.