Definition of ab testing

A/B testing, also known as split testing, is a marketing strategy that involves comparing two versions of a webpage, email, or advertisement to determine which one performs better in terms of conversion rates, engagement, or other desired metrics. This method allows marketers to make data-driven decisions by testing variations of design elements, copy, or calls-to-action, and analysing the results to optimise their campaigns. By identifying the most effective version, marketers can improve their overall marketing efforts, increase customer satisfaction, and ultimately boost their return on investment.

An example of ab testing

Here is an example of how it works:

Growth Method wants to test the effectiveness of two different pricing plans for their SaaS platform. They create two variations of their pricing page:

Variation A: A three-tiered pricing plan with a Basic, Pro, and Enterprise option.
Variation B: A two-tiered pricing plan with a Standard and Premium option.

They randomly assign 50% of their website visitors to see Variation A and the other 50% to see Variation B. Over the course of a month, they track the number of sign-ups for each pricing plan and the overall revenue generated from each variation.

At the end of the month, they analyze the data and determine which pricing plan variation led to higher conversions and revenue. Based on the results, they decide to implement the winning variation as their new pricing plan.

How does ab testing work?

AB testing works by comparing two or more variations of a marketing element, such as a webpage, email, or advertisement, to determine which version performs better. Marketers randomly assign their audience to different groups, each exposed to a different version of the marketing element. They then measure the performance of each version based on a specific goal, such as click-through rates, conversions, or sales. By analysing the results, marketers can identify the most effective version and implement it in their campaigns, ultimately improving their overall marketing strategy and return on investment.

Expert opinions and perspectives

Here are how some of the world's best marketing and growth professionals think about ab testing.

• "A/B testing is not only about the testing, but also about the learning." - Avinash Kaushik, Digital Marketing Evangelist at Google

• "Never stop testing, and your advertising will never stop improving." - David Ogilvy, Founder of Ogilvy & Mather

• "In God we trust. All others must bring data." - W. Edwards Deming, Statistician and Quality Control Expert

Questions to ask yourself

As a modern growth marketing or agile marketing professional, ask yourself the following questions with regard to ab testing:

1. What specific metric or goal am I trying to improve through this A/B test?

2. Are my test variations significantly different and relevant to the hypothesis I am testing?

3. Do I have a large enough sample size and test duration to ensure statistically significant results?

4. How will I analyse and interpret the results of the test to make data-driven decisions?

5. What are the potential risks or drawbacks of implementing the winning variation, and how can I mitigate them?

Additional reading

Here are some related articles and further reading around ab testing that you may find helpful.

1. Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing

2. The Ultimate Guide to A/B Testing

3. A/B Testing: The Complete Guide

4. The Ultimate Guide To A/B Testing

5. A/B Testing Mastery: From Beginner To Pro in a Blog Post

See how this topic is trending on Google Trends here: https://trends.google.com/trends/explore?date=all&q=ab%20testing

More questions? Connect with me on LinkedIn or Twitter, or book a Growth Call.