A/B Testing

A/B testing, also known as split testing, is a tool that site owners and business owners can use in order to manage their sales pages and landing pages and to optimize them to make the most possible sales. Theoretically, this is a ‘fool proof’ system that should turn even the least effective sales strategy into a honed system.

So how can it make such claims?

That’s basically because A/B testing acts almost like a form of evolution. This is survival of the fittest, wherein the least effective landing page or listing will ‘die out’ in order to make way for the strong.

A/B testing can be useful in a great number of different contexts, and this is something that we will examine as we cover the subject. That said, A/B testing is most often used in the context of internet marketing and sales. To begin with then, that’s what we will focus on.

Why You Need Split Testing – The Pain Point

In the context of making online sales, A/B testing works by creating two slightly different versions of the same website and then looking at how they perform.

Let’s say for instance that you are selling an ebook that promises to help you lose weight and get into shape. The way that you would normally go about doing this is to create a ‘sales page’ (also sometimes called a ‘landing page’) and fill that page with lots of text and images attempting to convince the reader that this is the book they have been looking for and that they need.

These are the long narrow pages that tend to say things like:

“Hey, are you sick and tired of looking overweight and feeling low on energy? I was like you once. I once believed that it was normal to feel that way as you get older. Heck, so many of my friends felt that way…”

You know the ones.

These pages are meticulously designed and crafted to be as highly effective as possible and as highly ‘optimized’ for sales. The aim is to increase the statistic known as the conversion rate. This is the number of people who open the page and then go on to buy.

Something like 0.1% is a low conversion rate, whereas 10% would be considered an incredibly high conversion rate. Seeing as you generally pay for clicks by using PPC advertising, you want to try and make the most of each of those clicks as possible by driving up your conversion rates. This will increase your profits.

Other tricks to increase your conversion rate include using a narrow layout (this encourages people to keep scrolling down), using lots of facts and figures along with social proof and testimonials, and using color tricks – such as the color red which is known to encourage impulsive action. It’s common to see lots of very short sentences, lots of bold text and underlined text and lots of headings.

The good news is that once you’ve designed your sales page, you aren’t ‘stuck’ with what you’ve built. The web works in such a way that you can continuously update and tweak the things you’ve put out.

So, if you build your landing page and find that it’s not converting any of your visitors into paying customers, then you can make changes in order to try and drive up that number.

Normally, you might do this by simply following your ‘gut’. Perhaps a bolder title would work? Maybe you need to increase the size of the font? Maybe you need to be more ‘in your face’ with the marketing talk. Or then again, maybe it’s too in your face and you need to be more subtle so as not to put off the more cynical visitors you get.

And herein lies the problem: there’s no way to be sure what the problem is or what the best thing to fix that problem would be.

Fulcrum tip: Therefore, you may well try to fix the problem by making a change, waiting to see what happens, and then making another change depending on how that goes. But this unsystematic and unscientific approach is highly vulnerable to errors and you can very easily end up making changes that actually hurt the conversion rate of your site and the quality in the long run.
Which is where A/B testing comes in!

How Split Testing Works
With split testing then, you instead take a more systematic approach to the problem and create two different versions of your site and upload these automatically.

So, you have one page where your ‘Buy Now’ button is red and one page where your ‘Buy Now’ button is blue.

This will then allow you to directly compare both pages and see which one does best out of the two. From there, you can then choose to adopt the one that brings in the most sales and you can forget the one that is less effective.

This is important because we can otherwise make the mistake of drawing conclusions. As they say in science, correlation does not mean causality. In other words, just because two things happen at the same time, that doesn’t necessarily mean that one is caused by the other.

Just because you changed your buy button to red and that increased the number of clicks, that doesn’t mean that red buttons drive more clicks! It could simply mean that you changed it on a Saturday and people are more likely to buy on a Saturday. Likewise, it might simply mean that by sheer random chance, you had more clicks on that day.

By doing a split test and seeing the two different versions performing side-by-side, you can rule out this possibility and you can be absolutely certain that it was this change that ultimately led to the improvement in performance.

A Word of Caution – You Need Statistical Significance
The very cynical among you may have already spotted a potential problem with this plan however. That is that there is still the possibility that any change will simply be caused by chance and not by anything you did.

Because if you have two separate websites and one is performing better than the other, that might just mean that the random selection of people that you sent to the other site happened to include more people who would buy from you!

So how do you avoid this mistake?

The answer is that you need to make sure that the data you are collecting is statistically significant. This is another term used in scientific studies and it’s very important that you take it into account.

Because if you have 100 visitors, 50 on each site, and 10 people buy from site number one versus none for site two… that could very well be chance.

However, if you have 1,000,000 visitors, 500,000 on each site, and you have 10,000 sales on one site and 53 on another… well then the odds of that being a coincidence are now dwindled to almost zero.

This number will never be zero. However, the longer you run the test, the lower you can make the number and the more certain you can be that you should adopt the change.

Testers also need to look out for other confounding variables. These are other factors that can unnecessarily complicate the results. For instance, if you have more success with your site with the red button and that is proven to be statistically significant, there is still an outside chance that there is a ‘confounding variable’ you haven’t thought of. For instance, maybe red is in fashion right now and this will only hold true until that fashion changes?

Or perhaps the red simply looks better with the design of button you’ve chosen. Maybe blue would be better if you changed the font.

You can never be 100% sure that you have removed all confounding variables. But what you can do is to keep all elements as precisely the same as possible in order to at least reducing the likelihood of these playing a role.

As you can see, A/B testing is not perfect and you always need to consider how you are using it and if it is indeed the right tool for the job at any given time.

How to Perform A/B Testing
So, the next question becomes: how do you perform A/B testing?

One option is to simply create two different versions of your landing page yourself and then to create a third page that will redirect visitors to one of those two sites. The redirect should be random and not based on geography or anything else like that. Ideally it will simply take it in turns. This will create a random sample and avoid sample bias (you don’t want all the early risers or people in one country being sent to one site for instance).

If you don’t have the technical skills to do this, then you can find plugins and tools that will handle the process for you. These include sites like Optimizely which will then allow you to see a calculation of the significance as well – making everything simple and easy for you.

The other option, regardless of which method you use to employ it, is to send only a very small portion of your audience to the two versions of your site. For instance, you have your main site and then you have your A/B test off to ‘the side’ where you will send a portion of your visitors.

Other Uses for A/B Testing
There are many more potential uses for A/B testing. For instance, as we’ve already discussed, this is a technique that is used in science all the time. For instance, in medicine if you were trying to test the effectiveness of a new drug, then you would normally split your participants into two groups. Group A will be given the medication and group B will be given a placebo. The results will then be tested for significance, in order to try and ‘prove’ beyond reasonable doubt that the cause for the observed effect was the new drug.

Fulcrum tip: Likewise, you can use A/B testing in order to test products and this also goes for beta tests of new products being given to focus groups. Apps are now using A/B tests more and more as well.
In any scenario, an A/B test allows the user to see which changes have positive effects and which don’t. That way, they can only adopt any new changes once they are 100% certain that they should be effective, allowing the product, service or sales page to gradually develop and get better and better over time.

In this way, the product, site, service or sales page can ‘evolve’ in order to become the most optimized version of itself – with little risk of failure.