Blog
The ABFinal Blog
Short, practical articles on A/B testing and conversion optimization, grouped by topic. Numbers in the examples come from our own calculators, so you can reproduce them.
A/B testing basics
- Is A/B Testing Worth It?A cost-benefit look at A/B testing, with the traffic thresholds where it starts to pay and what to do below them.
- Is A/B Testing a Strategy?No - it's a method. A strategy decides what to test and why; testing tells you whether you were right.
- The Benefits of A/B Testing (and What It Can't Do)Seven concrete benefits of A/B testing for online stores, plus the limits worth knowing before you start.
- What Are the Types of A/B Testing?Classic A/B, A/B/n, split URL, multivariate, A/A, bandits, server-side and price tests - what each is for.
- What Is A/B Testing, in Plain English?A/B testing explained without jargon, using a bakery, a coin and a store's checkout page.
- What Is the Difference Between A/B Testing and Split Testing?Usually nothing - the terms are mostly interchangeable. When people do draw a line, it's between in-page tests and split URL tests.
- What Is the Principle of A/B Testing?The core principle - randomized comparison - and the four rules that follow from it.
- When Should You Use A/B Testing?Test when the outcome is uncertain, the stakes are real and you have the traffic to detect the effect. Skip it for fixes and tiny tweaks.
- Why Do People Use A/B Testing?Because opinions about what works are wrong more often than anyone expects, and testing is the cheapest way to find out.
Statistics and analysis
- A/B Testing vs Hypothesis Testing - A Side-by-Side WalkthroughThe same idea in two vocabularies. A translation table and one test walked through in both languages.
- How to Analyze A/B Test Results, Step by StepA checklist you can follow at the end of every test, worked through with real numbers.
- Is A/B Testing Really Reliable?Reliable when run properly; misleading when underpowered, peeked at or broken. Here's what makes the difference.
- Is A/B Testing the Same as Hypothesis Testing?Not quite. An A/B test is an experiment that usually uses a hypothesis test to analyze it - but it also involves design, randomization and a business decision.
- T-Test vs A/B Test - What's the Difference?An A/B test is an experiment; a t-test is one way to analyze it. Here's when a t-test fits and when another test is better.
- What Are the Possible Outcomes of an A/B Test?Winner, loser, flat, inconclusive - and invalid. What each outcome looks like in the numbers and what to do next.
- What Does A/B Test Analysis Involve?Analysis is more than a p-value. The five layers - validity checks, the primary result, effect size, guardrails and segments - and what each tells you.
- What Is the Difference Between A/A and A/B Testing?An A/B test compares two different versions; an A/A test compares a version with itself to check your setup. Here's when to run one.
- What Is the Significance Test for A/B Testing?For conversion rates, it's usually a two-proportion z-test (or the equivalent chi-square test). Here's how it works and how to read it.
- Which Statistical Test Should You Use for an A/B Test?A decision table by metric type - conversion, revenue, counts, ratios - with the test that fits each and the traps to avoid.
Running tests well
- 12 Common A/B Testing Mistakes (and How to Avoid Them)The mistakes that produce false winners and wasted tests, from peeking and underpowered samples to broken splits and metric fishing.
- A/B Testing for Mobile ShoppersMobile visitors behave differently and convert lower. How to plan, build and analyze tests when most of your traffic is on phones.
- How Long Does It Take to Run an A/B Test?As long as it takes to reach the planned sample, rounded up to whole weeks - usually two to six weeks. Here's how to calculate yours.
- How Often Should You Run A/B Tests?As often as your traffic supports properly sized tests - which for most stores means a few per quarter per page type, not one a week.
- Price Testing Strategies for Online StoresSix ways to test prices - from a straight A/B price test to sequential tests, bundles and shipping thresholds - with the trade-offs of each.
Shopify and e-commerce CRO
- How to Create a High-Converting Shopify HomepageThe home page's real job is routing visitors to products. What to put on it, what to cut, and how to tell if changes help.
- How to Create a User-Friendly Navigation on ShopifyBuild menus around how customers shop, keep labels plain, make search and filters work, and test navigation changes carefully.
- How to Improve Shopify Conversion RatesWhere Shopify stores usually lose buyers, the fixes to make first, and the changes worth testing - in order of impact.
- How to Optimize Site Speed on ShopifyFind what's slowing your store, fix the usual culprits (apps, images, fonts, scripts), and check the results with real-user data.
- How to Optimize Your Prices on ShopifyKnow your margins, do the break-even math, choose a testing approach and judge results by profit per visitor.
- How to Use Retargeting Ads to Boost Shopify ConversionsSet up retargeting audiences by intent, match the message to where people dropped off, cap frequency and measure incremental sales with a holdout.
- How to Write SEO-Friendly Product Descriptions on ShopifyWrite descriptions that rank and sell - unique copy, the words customers use, scannable structure and the Shopify fields that matter.
- Optimizing Product Pages for Higher ConversionsWhat a converting product page needs, what to test on it, and how to size those tests on a typical store's traffic.