Run tests as often as your traffic allows each one to reach a meaningful sample size, and not more often than that. For most small and mid-sized stores, that means a handful of tests per quarter on each major page type, often run in parallel on different pages. Big platforms with millions of visitors run hundreds at once; that's a function of traffic, not ambition.
Work out your testing capacity
For each page type you'd test (product template, cart, checkout, home):
- Note monthly visitors and the conversion rate you'd measure there.
- Decide the smallest lift worth detecting.
- Use the sample size and duration calculators to get weeks per test.
If a product-page test takes four weeks, that page can host about 12 tests a year, minus holiday freezes. That's your cadence for that page.
Example: a store's product template gets 60,000 visitors a month at a 3% conversion rate. Over two weeks at 50/50, each group gets about 15,000 visitors, enough to detect lifts of roughly 19%. Over four weeks, roughly 13%. If the team's ideas are typically smaller than that, four-week tests (and about one test a month on that template) are the realistic rhythm.
Run tests in parallel, carefully
Different pages and funnel stages can each host a test at the same time:
- product page: gallery redesign,
- cart: shipping estimate display,
- checkout: express pay placement.
Because each test assigns users randomly and independently, a user's group in one test doesn't predict their group in another, so effects tend to balance out. Avoid parallel tests that could plausibly interact strongly, like two tests that both change the shipping message.
Don't speed up by cutting corners
The tempting shortcuts all make results worse:
- Stopping early when a result looks good inflates false positives.
- Shrinking the sample makes most tests inconclusive and exaggerates the winners you do find.
- Testing tiny changes to keep the calendar full produces effects too small to detect.
A steady pace of properly sized tests beats a frantic pace of noisy ones.
Keep the pipeline full
Cadence problems are often idea problems. Keep a backlog of researched hypotheses scored for impact, confidence and ease (the roadmap guide has a scorer), so that when one test ends the next is ready to build.
Pause around unusual periods
Black Friday, big launches, site migrations and stockouts distort behavior. Many teams freeze tests during these windows, or run tests specifically designed for them.
Review the program quarterly
Every quarter, look at the log: how many tests ran, how many were conclusive, what the winners have in common. If most tests are inconclusive, your tests are too small or your changes too timid, and running them more often won't fix that.
Common questions
Can I run multiple A/B tests at the same time?
Yes, on different pages or funnel stages where they're unlikely to interact, for example a product page test and a checkout test. Avoid two tests on the same element or page at once. With random assignment, overlapping tests on different pages usually spread evenly across each other's groups.
Should I always have a test running?
It's a good goal for high-traffic pages, because idle traffic is wasted learning. But a well-planned test next month beats a rushed one today.