Is A/B testing right for your website?

If you run a website, no doubt you’ve heard of A/B or split testing. This guide explains what A/B testing is, what to test and how it can help your sales.

What is A/B testing?

A/B testing or split testing is when different versions of a web page are shown to different website visitors to discover if a change increases conversions.

A/B testing is an important part of Conversion rate Optimisation (CRO).  It removes the uncertainty around making changes to a website when trying to increase conversions. A well run A/B test will give a definitive result that the changes have increased conversions.

A conversion can be a sale on an ecommerce site or it can be another metric, such as social shares or email signups.

There are many different types of A/B testing but usually 50% of visitors to a website will be shown the control or unchanged version of the page, the ‘A’ version. The other 50% will be shown the changed version of the webpage, the ‘B’ version.

Using A/B testing we can test ideas to increase conversions, or whatever business metric you are measuring. If the ‘B’ version results in an increase it proves the ideas for increasing conversions was correct. It can then be incorporated into the site permanently.

A/B testing is also a low risk way of making changes to a website. The change isn’t permanent and doesn’t have to be implemented if it proves to result in less conversions.

Some AB tests result in small increases in conversions. This is fine. What matters is the long term lift in revenue. A 5% increase per month in conversions will result in much more sales year on year.

What should I test?

Deciding what to test is the most difficult part of A/B testing. First you should have an idea of what could be preventing users from taking the desired action on your website. This article goes into more depth about CRO and creating a hypothesis for testing.

Testing can include:

• Copy changes. Headlines, website copy such and content, writing style and formatting.
• Website structure. Changing the layout of a website to move elements up or down and page. Adding items or removing items to reduce clutter.
• Navigation. Test different menu items and layouts.
• Forms. Changing the layout of forms to make them easier to use. Removing form inputs to make the form shorter.
• Add CTA’s (Call to action) Add CTA’s to web pages. Or change colours or copy.
• Social Proof. Adding reviews and testimonials.

Find out where your problems are on the key pages of your website. The pages where users take action. For ecommerce websites that would be product pages, cart and checkout.

Analytics and heatmaps can assist in finding problem areas on key pages of your website.

Make the most of the traffic to your ecommerce store with a CRO / UX audit.

How much traffic do I need?

A/B testing is better for more mature websites.

You’ll need 250 conversions for each variation at least, depending on your traffic. Ideally you want the test to run in about 4 weeks. Tests that run too long can be influenced by business trends that can affect the outcome. ie. Selling icecream in summer, but the test runs into autumn.

What if I don’t have enough traffic?

If you don’t have enough traffic to A/B test with conversions as a goal, you can test other KPIs. You might still have enough traffic to test add to cart rates.

When a website has some obvious problems the fixes can be implemented without testing.

A/B testing is 80% research and 20% running the test. The research phase can be valuable for improving your site. If you don’t enough traffic then use the research to make sweeping changes to you site.  This will give you enough of a swing in conversions to tell if the ideas for increasing conversions are in the ball park.

Read more about Conversion optimisation for low traffic websites here

Not every test will be a winner

Especially in the early days of starting a CRO program, not every test will be a success. This is because the CRO agency is getting to know how your business works and your audience.

All is not lost though. A losing test can be very informative. Just as a winning test gives us data on what works, a losing test lets us know what doesn’t work. Losing tests help to build knowledge of what doesn’t influence your users. Future A/B test designs can the be fine-tuned, and hopefully more successful.

Cool, how do I get started?

You will need a web developer that knows what they are doing. Quality assurance is a must when A/B testing. A poorly implemented test can be a waste of time, and at worst, screw up your site and cost you money.