Free tool · no sign-up
A/B test significance calculator
Enter visitors and conversions for both versions. Get the uplift, the p-value, a confidence interval for the difference and a verdict in plain words.
Your results
B wins. The difference is statistically significant at 95% confidence.
If A and B really converted the same, a gap this big would show up by chance only 4.7% of the time, less than the 5% your confidence level allows.
95% confidence interval for B − A
The true difference is likely between +0.01 and +1.79 percentage points. The interval doesn’t include zero.
How the verdict is reached
- Conversion rates: conversions ÷ visitors for each version.
- Uplift: (B − A) ÷ A.
- p-value: from a two-sided z-test on B − A, with the unpooled standard error of the difference.
- Confidence interval: B − A ± z × the unpooled standard error of the difference.
Significant isn’t the same as important
With enough traffic, a 0.1-point difference becomes significant and still not worth shipping. Look at the low end of the interval: if the smallest plausible gain is too small to matter, treat the result as a draw.
When to distrust a result
- Fewer than about 30 conversions in either version.
- The test was stopped the moment it turned significant.
- Many metrics or segments were checked and one came out significant.
- Traffic between versions is far from the split you set (a sample ratio mismatch).
Outcomify
Keep the evidence next to the decision it informed
Outcomify connects each experiment to the assumption it tested and the solution it was for, on your Opportunity Solution Tree, so the team remembers why you built what you built. Free plan, no credit card.