ABFinal

Pricing guide

Price A/B Testing for Online Stores

Price is one of the strongest levers a store has, and one of the easiest to get wrong. Here's how to test it without fooling yourself or upsetting customers.

Why price tests are different

Most A/B tests try to raise conversion. A price test expects conversion to change, usually downward for a price rise, and asks whether the trade is worth it. That changes three things:

  1. The metric. You judge on gross profit per visitor, not conversion.
  2. The math. You can calculate in advance how much conversion you can afford to lose.
  3. The risk. Customers who see two prices may feel cheated, and some jurisdictions regulate how prices can vary.

Break-even conversion: do this first

If your current price is P₁, the new one is P₂ and the variable cost per unit is C, the new price earns the same profit per visitor when:

CR₂ / CR₁ = (P₁ - C) / (P₂ - C)

Three examples:

Change Unit cost Conversion can change by Meaning
$40 → $45 $15 down to -16.7% price rise survives a big drop
$30 → $33 $12 down to -14.3% a 10% rise tolerates a 14% drop
$50 → $45 $20 must rise +20% a 10% discount needs 20% more buyers

The price test calculator does this for you and compares actual results with the break-even line.

Choosing price points

  • Test one step at a time. A 5% to 15% change is large enough to measure and small enough to be reversible.
  • Respect price thresholds. $49 and $50 can behave differently, and so can prices either side of your free-shipping threshold.
  • Pick products with enough volume. A price test on a product that sells five units a week will take months.
  • Consider testing the offer instead. Bundles, quantity breaks, shipping thresholds and guarantees change perceived value without two shoppers seeing two prices for the same item.

Sample size for price tests

Size the test on the conversion rate, using the drop you'd consider a failure as the effect. If you expect conversion to fall by no more than 10% and want to detect a 10% change, at a 3% baseline that's about 53,000 visitors per price. The sample size calculator gives the exact figure.

Profit per visitor is noisier than conversion because order sizes vary, so treat the conversion sample size as a floor.

Running the test

  • Keep prices consistent per visitor. A shopper who sees $45 on Monday shouldn't see $40 on Wednesday. Assign by user, not session.
  • Make every place a price appears consistent: collection pages, product pages, cart, checkout, emails and ads. A mismatch between the product page and the checkout is the fastest way to lose trust, and in many places it creates legal problems too.
  • Exclude existing subscribers and wholesale customers where prices are contractual.
  • Watch refunds and support tickets as guardrails.

Reading the results

Once the planned sample is in:

  1. Check the split with the SRM checker.
  2. Enter both prices in the price test calculator.
  3. Compare profit per visitor and where conversion landed relative to break-even.
  4. Look at the confidence interval for the conversion difference. If its pessimistic end falls below break-even, the new price could lose money and you need more data or a risk decision.

Random price tests are common in e-commerce, but rules differ widely. Some places require the price shown to be the price charged, restrict "was/now" comparisons, or limit pricing based on personal data. Platform policies may also apply. This page isn't legal advice: check the rules where you sell.

Separately, think about customer trust. A short, honest statement ready for anyone who asks, and a policy of honoring the lower price on request, cost little and prevent most problems.

Common questions

Is price A/B testing legal?

In many places random price testing is allowed, but it's regulated in ways that vary by country and state. Price display rules, consumer protection law, rules on personalized pricing and platform policies may all apply. Get advice for the markets you sell in before running a price test.

What metric should I use to judge a price test?

Gross profit per visitor. It combines conversion rate, order size and margin. Conversion rate alone will almost always favor the lower price, and revenue alone ignores costs.

How long should a price test run?

At least two full weeks and usually longer, because price effects can differ between first-time and returning customers and between paydays and the rest of the month. Use a sample size calculation for the conversion rate as a minimum.

What do I do when a customer notices a different price?

Decide in advance. Many stores honor the lower price for anyone who asks and explain that prices are being reviewed. Being caught out with no answer does more damage than the test can earn.

Are there alternatives to showing two prices at once?

Yes. You can test prices sequentially (one price for a few weeks, then another), test across comparable products, test bundles or shipping thresholds instead of the headline price, or use surveys such as Van Westendorp to narrow the range before testing.