For many online stores, most visitors arrive on phones. They browse in short bursts, tap rather than type, and convert at lower rates than desktop visitors. That changes what's worth testing and how much traffic you need.
Mobile conversion rates change the math
Lower baselines need bigger samples. Suppose a store converts 2.0% on mobile and 3.5% on desktop, and wants to detect a 15% relative lift:
| Device | Baseline | Visitors per group |
|---|---|---|
| Mobile | 2.0% | 36,693 |
| Desktop | 3.5% | 20,622 |
A mobile-only test needs almost twice the visitors per group, even though mobile may have more traffic in total. Run your own figures in the sample size calculator.
One test or two?
One test across devices is usually best. It uses all your traffic, and you can look at the device split afterwards as a secondary view (for ideas, not proof).
A mobile-only test makes sense when the change exists only on mobile: a sticky add-to-cart bar, a thumb-friendly menu, a different image carousel. Restrict the test to mobile users so desktop visitors don't dilute the result.
Avoid running a test on both and then declaring the device where it won. That's a segment-fishing trap.
What to test on small screens
Ideas that often matter more on mobile:
- Above-the-fold content. Price, variant selectors and the add-to-cart button visible without scrolling.
- Sticky add-to-cart that appears once the main button scrolls away.
- Image galleries. Swipe vs thumbnails, image order, a video in position two.
- Navigation. A visible search bar against a hamburger-only menu.
- Checkout friction. Express wallets placed higher, fewer fields, correct keyboard types.
- Speed. Every script and app you remove helps mobile most.
Build and QA with mobile in mind
Client-side testing tools change the page after it loads. On slow phones that causes flicker (the original flashes before the variant appears) and slows rendering, which can sink a variant that would otherwise win. Test variants on mid-range Android phones on mobile data, not just on a new iPhone over office Wi-Fi.
Measure across devices
Many shoppers start on a phone and finish on a laptop. If your tracking can't link the two, a change that makes mobile browsing better may show up as desktop conversions you can't attribute. Where possible, use logged-in or email-identified users for cross-device analysis, and consider add-to-cart or "save for later" as supporting metrics.
Apps are different
In native apps, changes ship through app store releases, so experiments usually run server-side: the app asks a server which variant to show. That makes it possible to change the split or end a test without waiting for a release.
For speed fixes that help every mobile visitor without any testing, see how to optimize site speed on Shopify.
Common questions
Should I run separate A/B tests for mobile and desktop?
If a change only makes sense on one device, yes. Otherwise run one test across both and check the device breakdown as a secondary look. Splitting a test by device halves the traffic for each and often leaves both underpowered.
Why do mobile visitors convert less?
Smaller screens, typing on touch keyboards, more interruptions and more browsing without intent to buy. Many shoppers research on a phone and buy later on a laptop, so mobile conversion understates mobile's role.
How do I A/B test a mobile app?
App tests usually run server-side or through feature flags, because app store releases are slow. The app asks a server which variant to show, so you can change the split without a new release.