An A/B test should run until it reaches the sample size you calculated before launch, and for at least one full week, ideally two. For most stores that lands somewhere between two and six weeks. The exact figure comes from four inputs: your traffic, your baseline conversion rate, the smallest lift worth detecting and how strict you want to be.
The calculation
- Work out the sample size per group with the sample size calculator.
- Multiply by the number of groups.
- Divide by the daily visitors who'll enter the test.
- Round up to whole weeks.
The duration calculator does all four steps.
Example. A product page gets 2,500 visitors a day and converts at 2.5%. You want to detect a 15% relative lift at 95% confidence and 80% power.
- Sample per group: 29,193
- Total for two groups: 58,386
- Days: 58,386 / 2,500 = 24
- Rounded to whole weeks: 28 days
What changes the answer most
| Change | Effect on duration |
|---|---|
| Halve the lift you want to detect | About 4x longer |
| Double the traffic | Half as long |
| Add a third variant | 1.5x longer |
| Raise power from 80% to 90% | About 1.3x longer |
| Lower confidence from 95% to 90% | About 0.8x as long |
| Only 50% of traffic in the test | 2x longer |
The first row is the one that surprises people. Small effects are expensive.
Why at least a week
Shopping behavior varies by day. Weekday lunch-break visitors, Sunday-evening browsers and Friday-night impulse buyers convert differently. A test that runs Tuesday to Friday is measuring an unrepresentative slice. Whole weeks keep the mix normal.
Two weeks also gives novelty a chance to wear off. Returning customers sometimes click on anything new simply because it's new, and that bump fades.
Why not much longer than six to eight weeks
- Cookie churn. People clear cookies or switch devices and get re-assigned, which blurs the groups.
- Seasonality. The baseline in week one may not match week eight.
- Site changes. New products, prices and campaigns accumulate during a long test.
- Opportunity cost. That's traffic you could use for the next test.
Don't stop early on a good day
The biggest timing mistake is checking daily and stopping the moment the result turns significant. Each check gives chance another opportunity to cross the line. In simulations with 14 daily checks, more than 20% of tests with no real difference show a "winner" at some point. Try it yourself in the A/A simulator.
Decide the end date before launch and look in the meantime only for problems: errors, broken pages, and a lopsided split, which the SRM checker catches.
If the duration is too long
- Test a bolder change.
- Test a higher-traffic page or a whole template.
- Measure a more frequent action, like add-to-cart, then check purchases after rollout.
- Use the MDE calculator to find what you can detect in the time you have.
Common questions
What is the minimum time to run an A/B test?
One full week, even if you reach the sample size sooner, so every day of the week is represented. Two weeks is safer because it smooths out an unusual week and lets novelty effects fade.
Can an A/B test run too long?
Yes. Past roughly six to eight weeks, cookie deletion re-assigns returning visitors, seasonality changes the baseline, and other site changes pile up. If a test needs that long, test a bigger change or a busier page.
Should I stop a test once it hits 95% significance?
Only if you've reached the sample size you planned. Stopping the first time it crosses 95% inflates false positives badly. If you want the option to stop early, use a sequential testing method designed for it.