t-test or ANOVA? Comparing groups
Comparing groups is the most common task in applied research, and also where one of its most common errors hides. The choice between a t-test and ANOVA — and what you’re allowed to do next — is worth getting exactly right.
Two groups, or more than two?
The dividing line is simple. Comparing the average of an outcome between two groups — treatment vs control, this year vs last — is the job of a t-test. Comparing three or more groups — say, three teaching methods — is the job of ANOVA (analysis of variance). Same underlying question — do these groups differ on average? — but the number of groups changes the correct tool.
Why not just run a t-test on every pair?
It’s tempting, with three groups, to run three t-tests (A vs B, A vs C, B vs C). Don’t. Every test carries a 5% chance of a false positive, and those chances compound: run enough comparisons and a “significant” result becomes almost inevitable by luck alone. This is the multiple-comparisons problem, and it is one of the most common ways real studies manufacture false findings.
Three groups don’t mean three t-tests. They mean one ANOVA — and only then, if it earns them, the follow-up comparisons.
The omnibus test, then post-hoc
ANOVA solves this with a two-stage logic. First the omnibus test asks a single question: is there any difference anywhere among the groups? Only if that overall test is significant do you proceed to post-hoc comparisons (like Tukey’s HSD) to find which groups differ — and those comparisons are adjusted to keep the overall error rate under control. Skipping the omnibus step and diving straight into pairwise tests is exactly the trap ANOVA exists to prevent.
Paired or independent?
Both tests have a paired cousin for repeated measurements. Measuring the same people before and after an intervention calls for a paired t-test; the same design across three or more time points calls for repeated-measures ANOVA. Pairing is worth seeking out: by comparing each person to themselves, it strips away individual differences and detects smaller effects with fewer participants.
Mind the assumptions
Both assume roughly normal data and similar variances between groups. When those fail, the non-parametric equivalents step in: the Mann–Whitney U for two groups, Kruskal–Wallis for several. Deciding which test applies, and whether its assumptions hold, is what our test selector is built for — and what thericerca checks against your real data before it computes anything.
The short version. Two groups: t-test. Three or more: ANOVA, then post-hoc only if the omnibus is significant. Same people measured repeatedly: the paired versions. Get that structure right and you’ve avoided the most common comparison mistake in applied research.