4 Steps To A Successful AB Test

May 9, 2022 | TIPS & TOOLS, PODCAST

Did you know that a CRO expert can easily pinpoint 50 to 150 problems on a website or mobile app? You can’t attempt to fix all your problems at once, so how should you prioritize? If you want to convert more customers and increase sales, the answer lies in AB testing, which is the cornerstone of any CRO program.

In this episode we discuss the four steps to running a successful AB test on your website or mobile app. This is the fourth episode in our series about CRO (Conversion Rate Optimization). If you’re new to CRO or would like a refresher on the basics of CRO, we encourage you to listen to our previous three episodes.


Listen In to Hear About:

  • A step-by-step breakdown of how we approach AB testing for our clients.
  • We cover the 4 steps for your AB experiment including:
    1. Problem identification – the framework for identifying problems in your marketing funnel using our 47-point CRO checklist
    2. Hypothesis – learn how to pinpoint the right KPIs, goals and structure for your testing program around it
    3. Measuring results – learn how to make decisions around statistically significant data
    4. Celebrate and repeat using our tips for success – even without looking at your website, we can tell you what you can fix TODAY to improve your conversion rate
  • Our giveaway exclusively for our podcast listeners: Apple Airpods


Resources mentioned in this episode:

Listen to the Full Episode

Connect with Digital Dames

[email protected]

Subscribe to the Podcast

4 Steps To A Successful AB Test

This is the fourth installment of our series about CRO (Conversion Rate Optimization).
We’ve talked a lot about AB testing and how it is the cornerstone of any conversion optimization program. Today, we are breaking down the steps for when you’re ready to launch a test on your website or app.

There are four steps we recommend: problem identification, building a hypothesis around the problem, measuring your results, and then celebrating and developing a new hypothesis. Read further for our top-level insights and listen to the full episode for more information and details.

Step 1: Problem Identification

You can identify opportunities for improvement for AB testing by looking at these seven key areas:

  • Personas development – how to create effective buyer personas that work
  • Trust and confidence – how to make sure your website elicits trust through design, copy, and testimonials.
  • Common Objections – consider the fears, uncertainties, and doubts that your consumer may have.
  • Incentives – How to test price, scarcity, and urgency on your site
  • Engagement – Where are there opportunities for engagement and how can you test them?
  • Buying stage –
     – Attention stage: Need recognition
     – Interest stage: Information search
     – Desire stage: Evaluation of alternatives
     – Action stage
     – Post action stage
  • Sales complexity – How fast a potential customer moves in a buying funnel defines the length and complexity of the sales process.

Using this Framework, a CRO expert can easily pinpoint 50 to 150 problems on a webpage. You should not attempt to fix all of these at once and instead prioritize and focus on the top three to seven problems to get started.

Step 2: Build Your Hypothesis Around the Problem

Hypothesis: You will need to have a specific question in mind, with a specified outcome or goal in mind. This should also be relevant to the marketing funnel and the buyer’s journey.

KPI: This is the unit of measurement that you’ll use to measure the outcome of your testing. Ideally will include the conversion percentage.You should typically be dividing the number of actions by the number of visitors.

For example:
 – Hypothesis: By featuring more client reviews and media logos on our homepage, we will increase clicks from the home page to the shop all page.
 – Goal: Increase clicks to the shop all page
 – KPI: Conversion Rate on button

Another example is:
 – Hypothesis: By showing our return policy on our cart page, we will improve checkout rates.
 – Goal: Checkout rates increase.
 – KPI: Improvement of checkout conversion (from cart)

Step 3: Measure Results

Statistical significance is what we like to refer to as “the magic math of marketing.” Statistical significance is the likelihood that the difference in conversion rates between a given variation and the baseline is not due to random chance. This is important because it helps us know from a probability perspective that our results are usually going to turn out this way. If you don’t have statistical results, you’re basically throwing darts in the dark. We have a calculators on our website that can help with this:

We also have a calculator that can help you to determine how long you have to run your experiment based on your traffic in order to get a statistically significant result.

Step 4: Celebrate and Develop a New Hypothesis

Once you improve your primary funnel, you can focus on different areas of marketing such as:

  • a different customer persona
  • a specific product, product category or service area
  • a different part of the buying journey
Our Favorite CRO Tips:

Without even seeing your website, we can almost guarantee that by adding these to your website you will improve conversions:

  • Above the fold hero – make it conversion-focused
    Pretend that no one is going to scroll past your hero image, which most do not. 57% of view time is spent on the Hero section, which is the oversized image at the top of the page.
    Evaluate your website and ask yourself:
     – Is your copywriting BENEFITS driven vs. FEATURES driven?
        A solar company’s headline:
        “Go solar with us today” vs “Save $20,000 in electric bills with solar”
     – Is your call-to-action clear?
     – Are you using images that are emotionally compelling and representative of your target audience?
  • Reduce cognitive overload – focus on a single goal.
    Choice overload, also known as overchoice, choice paralysis, or the paradox of choice, describes how people get overwhelmed when they are presented with a large number of options to choose from. While we tend to assume that more choice is a good thing, in many cases, research has shown that we have a harder time choosing from a larger array of options.
  • Testimonials – Make them bigger, and add them to your checkout flow.
  • Social Proof – Be more strategic about how others are either using your product, or rating your service.
You can get our full 47-point CRO checklist to help you identify areas of opportunity for your AB testing program here:


Check back each week for more MarTech tips, insight, news, and more.


Airpod Giveaway

Now, who wants some airpods!? We are giving away airpods exclusively to some of our podcast listeners! To enter, click here.

Airpods Giveaway

We hope you enjoyed reading this blog post

If you'd like our team at Digital Dames to help you massively improve website traffic and conversions, just book a call.

Post Contributors

Jaclyn Hawtin

Jaclyn Hawtin

Senior Data Architect

Over a decade of experience in product management, devops, startups, and agile methodologies. Track record of simplifying complex technical processes for cross-functional teams. Proficient in user centered design, UX, IX, UI, IA, user research and data analytics for responsive web, mobile and tablet applications. Incredibly adaptable, fluent with both people and machines.

Mani O'Brien

Mani O'Brien

Conversion optimization manager

Mani is a senior marketing manager with roots in storytelling. She nerds out on everything data, technology, human behavior and design. Chat with her about UX/UI, marketing funnels, conversion and goal tracking, marketing experimentation and astrology (she’s a Virgo Sun, Aries Rising).

Most Read
What is a Dimension in Google Analytics

What is a Dimension in Google Analytics

What is a Dimension in Google Analytics?Understanding the data collected by this powerful tool is essential for anyone looking to optimize their online presence and drive their business forward. But with so much information available, it can be overwhelming to know...

How to merge 2 users in mixpanel using postman?

How to merge 2 users in mixpanel using postman?

Postman is a great tool to use for merging duplicate profiles in Mixpanel Sometimes when working with Mixpanel you run into an issue where you have duplicate profiles. Luckily Mixpanel has an API endpoint for this. One of the easiest solutions is to connect your...

How to export a gif from figma?

How to export a gif from figma?

To export a gif from Figma, follow these steps: Open the Figma project that contains the design you want to export as a gif. Select the frame or frames that you want to include in the gif. You can do this by clicking and dragging your mouse to create a selection box...

Free Analytics 101 Course

Sign up for free access to start your data journey
First Name
This field is for validation purposes and should be left unchanged.
Conversion Path Ep. 12 -- Shopify Conversion Tips With Guest Brad Redding from Elevar Ep. 12 -- Shopify Conversion Tips With Guest Brad Redding from Elevar More