Building a CRO Testing Roadmap

The scorer above re-sorts as you type. Replace the examples with your own ideas.

Step 1: Count your testing slots

Before collecting ideas, work out capacity. For each page type you might test (product page, cart, checkout, home), note monthly visitors and the conversion rate you'd measure. Then use the MDE calculator to see what you can detect in two or four weeks.

For example, a product template with 60,000 visitors a month and a 3% conversion rate gives about 15,000 visitors per group over two weeks with a 50/50 split, enough to detect a lift of roughly 19%. Over four weeks, about 13%. That's your bar.

Step 2: Collect ideas from evidence

Pull ideas from funnel data, recordings, surveys, support tickets and usability tests. Write each as a hypothesis: the change, the expected effect and the reason. The CRO audit checklist is a good way to find the obvious gaps first.

Step 3: Score and sort

Score each idea with ICE, or a variant like PIE (Potential, Importance, Ease). Two cautions:

• Scores are only as good as the conversation behind them. Have two people score independently and discuss the differences.
• Big-impact ideas often score low on ease. Don't let a backlog of easy, small tests crowd them out forever.

Step 4: Filter by detectability

Drop or merge ideas whose realistic effect is far below your MDE. "Change the button text" might move conversion by 1%. If you can only detect 15%, it will never produce a clear result. Bundle it with other changes or just ship it.

Step 5: Build the calendar

Lay out tests per page type, one at a time on each, with the planned duration from the duration calculator. Leave gaps for holidays and big campaigns, which distort results.

A quarter might look like this:

Step 6: Review every month

Record results in a test log. Re-score the backlog in light of what you learned: a win on shipping transparency should raise confidence in related ideas, a loss should lower it.