How to choose the right statistical test
Faced with a fresh dataset and a research question, most people reach for the test they remember best. But the correct test isn’t a preference — it’s the logical consequence of what you measured and how. Here’s how to derive it in a few clear steps.
Start with the question, not the test
The most common way an analysis goes wrong is that it starts at the end — “let’s run a t-test” — instead of at the beginning: what are you actually comparing, and what kind of data do you have? The right test is not a matter of taste or habit. It follows, almost mechanically, from three properties of your question and your variables.
The three things that decide the test
1. The measurement level of each variable. Is it categorical (a group label like “treatment” vs “control”), ordinal (a ranked scale), or continuous (a real number like reaction time or GPA)? A relationship between two continuous variables calls for correlation or regression; a relationship between two categorical variables calls for chi-square.
2. How many groups or variables are involved. Comparing an outcome between two groups points to a t-test; three or more groups point to ANOVA. Relating several predictors to one outcome points to multiple regression.
3. Whether the observations are paired or independent. Measuring the same people twice (before and after) is a paired design; measuring two separate groups is independent. The paired version of a test is almost always more powerful, because it removes the person-to-person variation.
The right test isn’t chosen. It’s derived — from the level of your data, the number of groups, and how they were measured.
A quick map
- Two continuous variables, is there a relationship? Pearson correlation (or Spearman if the data isn’t normal).
- Two groups, different means? Independent-samples t-test.
- Same people measured twice? Paired t-test.
- Three or more groups? One-way ANOVA, then post-hoc comparisons.
- Two categorical variables? Chi-square test of independence.
- Predicting a continuous outcome from several variables? Multiple regression.
Assumptions decide the final form
Picking the family of test is only the first step. Each test carries assumptions — normality, equal variances, independence — and when they fail, the honest move is to switch to a version that doesn’t require them. A t-test on non-normal data gives way to the Mann–Whitney U; ANOVA under unequal variance gives way to Kruskal–Wallis. Choosing a test and checking its assumptions aren’t two separate jobs; they’re the same job. We cover it in parametric vs non-parametric tests.
Let the rules do the work
If you’d rather not hold this whole decision tree in your head, that’s exactly what our free which-test tool is for: answer a few plain questions about your variables and it names the correct test and its assumptions. And when you’re ready to run the analysis for real, thericerca makes the same choice in the open, checks each assumption against your data, and computes every number in Python.
A guide, not a substitute. This map covers the common cases; unusual designs (nested data, repeated measures, non-linear relationships) deserve a statistician’s eye. The goal is to get the everyday choices right — and to recognise when your case is genuinely out of the ordinary.