Our Top Tips For Setting Up A/B Split Tests

At Pop Creative, we have extensive experience in running A/B and multi-variant tests. Our team of conversion rate specialists combine their knowledge of how to create a great user experience to come up with new variants that increase conversions.

An A/B split test is simply a test that is run on a website between the original and a variant. Traffic is usually split equally between the two versions so a fair result can be produced. Variations can include anything from a very simple design or copy change to a whole redesign of a page (or pages).

The main purpose of a split test is to ultimately improve conversions on a website. The good thing about split testing is that you get to have an insight into whether the change will actually have a positive impact before making the change permanent. Whether you are going to make a small or a big change, it can all lead you to an increased conversion rate.

Another benefit of running split tests, rather than just making the change straight away, is that you remove external factors that could sway the results such as; seasonal trends, marketing campaigns and any other changes that have been made to the site in that time.

We have put together a list of our top tips that you should think about before you set up an A/B split test.

1) Don’t have any assumptions

Before beginning a split test, it is important to have an open mind and not try to predict the results. It is bad practice to assume that because a particular design element works for one website, that it’s going to work for every website. Start your test with a hypothesis, but be prepared for the result to go ways that you may not expect.

2) Analyse the needs of your audience

Websites with high conversion rates will be the websites that know their target audience well. The idea of split testing is to test whether a certain element will attract your audience more. Colours, wording, button size, and the way something is laid out on the page are all elements that need to be considered. Carry out extensive research into your demographic and what they expect to see on a website. It is also a good idea to discover what issues your users may be experiencing with the current website. You can then address this within the test.

3) Run the split test for long enough

You will need to run an A/B split test until it reaches statistical significance. Firstly, this means testing a large enough sample size. Depending on the amount of traffic you get through to your website, you should run the test until a large percentage of your monthly traffic has viewed both of the versions. Using a split testing software like Google Optimize, will help you hugely when deciding on ending a test. It will let you know when a test has drawn a conclusion and has been running for a sufficient amount of time. A split test should ideally be run for a minimum of a month, to be able to fairly compare users that visit at different times of the month.

4) Keep the test running

The most important thing to do when running a split test is to let it run. No matter how you think the test is going, leave it alone. Keeping a close eye on a test is recommended to notice any potential tracking issues which might invalidate the results, but don’t be tempted to end the test early if it’s not going the way you expected it to. It won’t be a fair test, and you won’t learn anything from it.

5) Decide the value of the test

You should decide the value of the split tests that you want to run, and the time or cost of implementing each one. You may have some ideas that will almost guarantee a clear win, but the cost and time needed to put the test together might be too high for the potential return. Smaller incremental tests are better, where possible, so it is easier to determine the value of each element being added.

At Pop Creative, we run A/B split tests for a range of industries with both e-commerce and marketing websites. We have extensive knowledge into how, and what to test which provides us with the ability to run carefully thought out split tests. We can then implement the changes for you too.
If you are interested in our monthly packages, email us at studio@popcreative.co.uk

View More

Contact Us

The Old Smithy,
Church Street,
SY11 2SP

01691 662712

Privacy Policy