What the calculator does
You give it four things: your current conversion rate, the smallest change worth detecting, how strict you want to be about false positives (alpha), and how likely you want to be to catch a real effect (power). It returns the number of visitors each group needs.
The formula is the standard normal approximation for comparing two proportions, the same one R's power.prop.test uses:
The result is rounded up to a whole visitor. We check it against reference values on every build; for example, p₁ = 20%, p₂ = 25%, alpha 5%, power 80% gives 1,093.74, matching R. The methodology page lists the rest.
A worked example
A store converts 3% of product page visitors. The team wants to know if a new product gallery lifts that by at least 10% in relative terms, so from 3.0% to 3.3%.
- Baseline: 3%
- MDE: 10% relative
- Alpha: 5%, two-sided
- Power: 80%
Answer: 53,211 visitors per variant, 106,422 in total. If the page gets 3,000 visitors a day, that's about 36 days. Most teams look at that number and decide to test a bolder idea, or to run the test on all product pages at once rather than a single template.
Why small effects are so expensive
Sample size grows with the inverse square of the effect. Detecting a 5% lift at the same 3% baseline needs 207,938 per group, almost four times the 10% figure. A low baseline makes it worse: at 2%, a 10% lift needs 80,682 per group.
This is why most stores can't run dozens of tiny copy tests and expect clean answers. The traffic isn't there. Put the effort into changes big enough to move the number.
Counting visitors correctly
Use the number of people who could actually see the change. If you test a checkout step, count visitors who reached checkout, and use the checkout conversion rate as the baseline. Plugging sitewide sessions into a checkout test inflates the baseline traffic and makes the test look faster than it will be.
Use users, not page views, if your testing tool assigns variants per user. Someone who reloads a page five times is still one person in one group.
After you have a number
Turn it into a calendar with the test duration calculator. If the duration is too long, use the MDE calculator to see what you can detect with the traffic you have. And decide the stopping point now: the A/A simulator shows what happens to your error rate if you stop the first time the result looks good.
Common questions
How many visitors do I need for an A/B test?
It depends mostly on your baseline conversion rate and the smallest lift you want to detect. At a 3% baseline, detecting a 10% relative lift (3.0% to 3.3%) at 95% confidence and 80% power takes about 53,211 visitors per variant. Detecting a 20% lift takes about 13,914. Halving the effect roughly quadruples the sample.
What is a good minimum detectable effect?
Pick the smallest change that would be worth shipping, not the change you hope for. For checkout or pricing tests on a store with modest traffic, 10% to 20% relative is common. If the required sample is out of reach, the honest options are a bolder change, a higher-traffic page, or a longer test.
Should I use 80% or 90% power?
80% is the usual default and means a real effect of exactly your MDE is missed one time in five. Moving to 90% costs about a third more traffic (71,233 instead of 53,211 per group in the 3% example). Use 90% when missing a real winner is expensive.
Does a one-sided test need fewer visitors?
Yes, about 20% fewer at the same alpha. In the 3% example it drops to 41,914 per group. The catch is that a one-sided test cannot tell you a variant is worse, so only use it when you'd treat "worse" and "no difference" the same way.
How do I handle more than one variant?
Every extra variant needs its own group of the same size. If you compare each variant with control and want the overall false-positive rate to stay at 5%, tick the Bonferroni option, which divides alpha by the number of comparisons. With three variants and a control the per-group requirement in the 3% example rises from 53,211 to 70,974.