5 A/B Testing Mistakes: How to Avoid Making Them and Increase Your Conversion Rates

A/B Testing, also known as split testing, is a powerful method for optimising user experiences, conversions, and other key metrics.

A/B Testing is a fantastic, cost effective method for increasing your conversion rates. However, there are many businesses that stumble into avoidable pitfalls when conducting A/B tests on their website. Today we’ll be looking at 5 common mistakes that online businesses make and provide some practical advice on how to enhance your A/B testing process.

1. Believing that A/B Tests should only be used on a Landing Page

One very common misconception about A/B testing is that they are only used to test landing pages on a website. This is definitely one of the primary reasons to use an A/B test, but certainly not the only application. A/B testing can be used on many pages across your website and run experiments on a variety of elements. Some of these include:

• Call to actions buttons and content
• Product imagery and visuals
• Descriptions of services & features
• Font sizes and styles
• Element placing and scaling

Remember that while navigating through your website, potential customers will have to transition through a variety of different pages, including product pages, service pages and your checkout or contact page. A/B testing all of the customer journey will help you make better decisions for your entire website based on the results.

2. Forgetting about external factors and seasonal changes

Website traffic is not equally steady across the entire year. Quite often there are other factors that influence your traffic and visitor behaviour. This is quite easy to see from something like Google Trends. For example, looking over the past 5 years for a product such as “Face Masks”, it is easy to see the COVID-19 Pandemic’s influence on behaviour and search volumes.

Regular seasonal changes and events, such as Christmas, Easter, the Summer season can also drastically change how visitors interact with your website and services. Failing to consider these can significantly skew the results of an A/B test.

Before running any kind of A/B test, think about what else is going on that could impact the results. Are your competitors running a promotion? Are you running a promotion on a different product or service? Is a specific product popular (or unpopular!) at the moment? Remember to consider these kinds of influences when interpreting your results.

3. Running too many A/B tests at once

While time available for testing is always important, it may seem like a great idea to run multiple A/B tests at once. This is certainly possible, however a common mistake is to conduct multiple tests at the same time that can interfere with each other’s results. If an A/B test can influence the performance of another, attributing which changes were actually meaningful in both tests can become very difficult, leading to frustration and wasted time.

If you wish to conduct multiple tests or test a variety of elements simultaneously, consider using multivariate testing. Alternatively, set up each of your A/B tests separately and run them one at a time, so that each time you get a set of results, you’ll know for sure it was only from the effects of that particular A/B test. This way you can make an informed decision about a change before beginning your next A/B test.

4. Using A/B Testing rather than Multivariate Testing

An A/B test is just that, a comparison between variation A and variation B. While this comparison sounds straightforward, it can get increasingly confusing to measure the impact of an individual change if your variations are significantly different.

For example, if running an A/B test with the following changes to variation B:

• The Primary Call-To-Action button changed from red to green.
• Bigger, bolder font across the entire page.
• Split up paragraphs for readability.
• Hid extra features and details behind clickable tabs.
• Added a live-chat option to the page.

After running the test, variation B with the above changes got 10% more conversions. Which change caused this? Was the added live-chat really effective but the bolder font and button changes detrimental? A single A/B test may not be able to provide you with all the answers.

As we mentioned above, for running a test on a large variety of elements, multivariate testing can run many different variations of a page at once and are more suited for complex testing of a web page and provides much better reporting of user interactions between the different variations being tested.

5. Running an A/B test without enough users available

We all want to know if we can make our websites better. However a common mistake when running A/B tests is not having a large enough number of users to get a meaningful result. A web page can have low traffic for many reasons, some of which you may be trying to figure out with the A/B test itself. Running a test with too few visitors can ultimately be detrimental, by giving you inaccurate results.

Performing an A/B test without a significant number of users to be statistically relevant can cause mistakes, if deciding to implement changes based on those results.

Running an A/B test is a process that takes time and for pages with low traffic, this time may be longer than otherwise expected. However you can supplement your A/B tests by pushing more traffic towards a particular page, perhaps by running email marketing or PPC paid ads to push traffic to the page you want to test. Just remember to keep these traffic sources in mind as well as factors that could influence the results!

Conclusion

Every website out there is unique, so when coming up with A/B testing ideas for your online business, remember to perform meaningful tests to help you decide whether or not a change is right for you. A/B testing is a fantastic way to build improvements into your website over time, and even if a new variation makes no improvement, you’ll learn more about your visitors and their behaviour by running such tests. Provided you don’t make these mistakes when setting them up!

If you need help running A/B tests on your website to improve your conversion rates, why not consider our conversion rate optimisation service? If this is something you think we can help you with, get in touch with us today!