Read-only sample study. A complete walkthrough — from problem statement to a validated report — using onboarding_ab.csv (200 rows × 10 cols). Sign in to run your own.

Sample study · comparative design

Onboarding Redesign A/B Test

Research information

Statement of the problem

This UX study compares two onboarding designs (variant A vs variant B) on time to first value, task completion, and self-reported satisfaction using a 5-item SUPR-Q scale, with 200 users randomly assigned across the variants.

Objectives

To describe time_to_first_value_seconds, completed, and suprq_total across the two onboarding variants.
To determine whether time_to_first_value_seconds differs between variant A and variant B.
To determine the association between completed and variant.
To assess the reliability of the SUPR-Q items suprq_1, suprq_2, suprq_3, suprq_4, and suprq_5.
To determine whether suprq_total differs between variant A and variant B.

Dataset

onboarding_ab.csv200 rows × 10 cols

Column	Kind	Distinct
user_id	text	200
variant	categorical	2
time_to_first_value_seconds	numeric	93
completed	numeric	2
suprq_1	numeric	5
suprq_2	numeric	5
suprq_3	numeric	5
suprq_4	numeric	4
suprq_5	numeric	5
suprq_total	numeric	19

Feasibility & estimateFree

✓ Feasibledesign: comparative · via heuristic

Detected a comparative design with 9 analyzable variable(s) and 5 analytical question(s). Recommended 6 statistical test(s). Estimated cost: 63 credits.

Recommended analysis

To describe time_to_first_value_seconds, completed, and suprq_total across the two onboarding variants.

Descriptive Statistics(time_to_first_value_seconds, suprq_total)

Frequency & Percentage Distribution(variant, completed)

To determine whether time_to_first_value_seconds differs between variant A and variant B.

Independent-Samples t-Test(time_to_first_value_seconds, variant)

To determine the association between completed and variant.

Chi-Square Test of Independence(variant, completed)

To assess the reliability of the SUPR-Q items suprq_1, suprq_2, suprq_3, suprq_4, and suprq_5.

Cronbach's Alpha(suprq_1, suprq_2, suprq_3, suprq_4, suprq_5)

McDonald's Omega(suprq_1, suprq_2, suprq_3, suprq_4, suprq_5)

To determine whether suprq_total differs between variant A and variant B.

Independent-Samples t-Test(suprq_total, variant)

Dataset Complexity15

Statistical Tests20

AI Interpretation15

Visualizations8

Report Generation5

Total to run63 credits

Estimated runtime: ~1-2 minutes

ResultsComputed

7 tests63 credits

Descriptive Statistics (time_to_first_value_seconds, suprq_total)

n	max	min	std	mean	mode	range	median	variable	variance
200	251	32	33.296	90.260	81	219	86	time_to_first_value_seconds	1108.626
200	25	7	3.906	17.825	15	18	18	suprq_total	15.261

Frequency & Percentage Distribution (variant, completed)

variant · A50.0%

variant · B50.0%

completed · 178.5%

completed · 021.5%

Mann-Whitney U Test (time_to_first_value_seconds, variant)

significant

p< .001

Rank-biserial r0.408

medium

n	std	mean	group
100	35.865	102.260	A
100	25.519	78.260	B

The result is statistically significant (p = 0.0000 < 0.05).

Mann-Whitney U Test was used instead of the Independent-Samples t-Test because the latter's assumption(s) were not met (Normality of group 'A' (Shapiro-Wilk), Homogeneity of variance (Levene)).

Chi-Square Test of Independence (variant, completed)

significant

χ²7.584df1N200p.006

Cramér's V0.195

small

0	1	row
30	70	A
13	87	B

The result is statistically significant (p = 0.0059 < 0.05).

Cronbach's Alpha (suprq_1, suprq_2, suprq_3, suprq_4, suprq_5)

α0.924N200

Internal consistency is excellent (alpha = 0.924).

McDonald's Omega (suprq_1, suprq_2, suprq_3, suprq_4, suprq_5)

N200

item	loading
suprq_1	-0.864
suprq_2	-0.826
suprq_3	-0.825
suprq_4	-0.849
suprq_5	-0.849

Internal consistency (McDonald's omega) is excellent (omega = 0.924).

Mann-Whitney U Test (suprq_total, variant)

significant

p< .001

Rank-biserial r-0.500

medium

n	std	mean	group
100	3.515	16.160	A
100	3.566	19.490	B

The result is statistically significant (p = 0.0000 < 0.05).

Mann-Whitney U Test was used instead of the Independent-Samples t-Test because the latter's assumption(s) were not met (Normality of group 'B' (Shapiro-Wilk)).

Report

Publication-ready report

Tests used · Results · Discussion — generated from the computed statistics.

Every claim checked · 30 numbers verified · 3 findings cross-checked · prose: llm:gemini

Open report ↗Download DOCX Download PDF

Reproducible code

Download the exact Jupyter notebook or Python script that reproduces these results with widely-used libraries (scipy, statsmodels, lifelines).

Download Jupyter notebook Download Python script

Run your own study

Bring your problem statement, objectives, and a dataset — thericerca does the rest.

Get started →