Statistics for UX research: the quantitative workflow, in one place
Quantitative UX research has a compact toolkit — task metrics, satisfaction ratings, A/B comparisons, and the occasional scale-reliability check — but the studies are frequent, the sample sizes are small, and the report turnaround is tight. This post maps that daily workflow onto what thericerca does, so you can see whether the two fit.
What quant UX statistics actually looks like
The typical quant UX study is small, fast, and comparison-driven: does variant B outperform variant A on task completion, error rate, satisfaction, or NPS? Does one persona segment behave differently from another? Is our survey scale reliable enough to trust week-over-week? The methods are the same handful of tests every time — but choosing the right one on a real dataset (paired vs independent, parametric vs non-parametric, when to control for baseline) is where mistakes happen fast.
Quant UX doesn't fail from methodological ambition. It fails from a paired t-test where an independent one belonged, and a satisfaction average that never got a confidence interval.
Your typical study, mapped
- Task time / error count between two design variants — Welch's t-test (times are usually skewed, so Mann-Whitney U is often the safer fallback — thericerca will make that swap for you if normality fails).
- Completion rate between variants — chi-square, or Fisher's exact for small samples.
- Within-subjects comparison (same user, two designs) — paired t-test / Wilcoxon signed-rank.
- Three or more designs (concept test) — one-way ANOVA / Kruskal-Wallis, with pairwise post-hocs.
- SUS / SUPR-Q / other satisfaction-scale reliability — Cronbach's α and McDonald's ω, plus an item-drop table.
- What predicts satisfaction (behavior + demographics) — multiple linear regression.
- Persona segmentation on scale items — exploratory factor analysis (EFA) to find dimensions before clustering.
- NPS change over releases — chi-square (promoter / passive / detractor) or ANOVA on the raw score.
The workflow, cell by cell
- Export the study data as CSV — from your testing platform, survey tool, or product analytics.
- Enter the research question and objectives — thericerca will pick the right test per objective and check assumptions.
- Post-hocs (Tukey, Games-Howell, Dunn's) fire only when their omnibus was significant, so you don't over-claim.
- Download the report in HTML/DOCX/PDF for the stakeholders + a Jupyter notebook for the archive.
For lightweight one-off checks that don't need a full study, three of our free tools cover the most common asks with no sign-up: the test-picker answers "which test do I use here?", the sample-size calculator tells you what N you'll need for a given effect and power, and the reliability tool gives you Cronbach's α with the item-drop table in seconds.
When you'd need something else
Beyond thericerca's scope. Large-scale product experimentation with sequential-analysis stopping rules (Optimizely, Statsig, GrowthBook, Eppo). Bandit / multi-armed algorithms. Bayesian A/B testing with informative priors. Long-run causal-impact estimation. Deep-behavioral funnel analysis (Amplitude, Mixpanel). thericerca is the right tool for study-shaped work; a growth-experimentation platform is the right tool for always-on product testing.
A concrete example
A/B test of an onboarding redesign: 200 users randomly assigned to variant A or variant B, measured on time-to-first-value (seconds), task completion (yes/no), and a 5-item SUPR-Q satisfaction score.
- Time-to-first-value — Welch's t-test (times are typically right-skewed; if normality fails, thericerca swaps in Mann-Whitney U with a note explaining why).
- Completion rate — chi-square with Cramér's V.
- SUPR-Q score — Cronbach's α on the 5 items, then a t-test on the sum.
- Report — an APA-formatted write-up you can drop into the design review, plus the reproducibility notebook for the data-team archive.
The short version. If your quant UX work is small-sample and study-shaped — usability testing, concept tests, satisfaction tracking, scale validation — thericerca covers it end-to-end. For always-on product experimentation, a growth platform is the right home.