A/B testing forms the back bone of conversion rate optimization. This episode will teach you about the basics and common pitfalls to avoid when you embark on your A/B testing adventures.
Need the TL;DR (or TL;DWatch) Version of this Episode?
- The number 1 tool in a Conversion Rate Optimizer’s tool belt is the A/B test. An A/B test creates two versions of a digital asset (webpage, form, pop-up, etc)—A and B—and shows how a sample audience responds to them in real-time. Simply put: it takes the guesswork out of website optimization. It is the most effective way to learn what your audience best responds to
- The Obama experience: One of the most powerful examples that shows the impact of a/b testing Obama’s 2008 campaigning efforts. In 2007, Obama was new to the presidential race, trailing by double digits in the polls and struggling to raise money to keep things moving. With their backs against the wall, Obama and his campaign team couldn't afford to leave any donations on the table. The director of analytics recommended running an experiment to test different combinations of website images and donation buttons to find the most effective way to encourage sign-ups and contributions. They started testing four different buttons and six different media on the primary landing page to find a winning combination. After running 24 unique combinations against each other to more than 300,000 website visitors, the team finally had the results of their experiment
- The 4 keys to an effective A/B test: To run an effective A/B test you need to identify your control variables, choose the right tools, outline meaningful goals, and ensure statistical significance.
- Identify your control variables: It’s crucial that you don’t completely redesign the digital asset for your test (Unless your goal is to just see if one webpage is superior in general to another.) Instead, change just one thing in the asset, and run the A/B test. You could, and should, change and test all sorts of things. Capitalized letters vs. regular sentence case, different types of images, different colors on your CTA button, and of course different copy in your value proposition. By identifying a single element per test to change you’ll learn over time what works best to drive conversions.
- Choose the right tools: From Google Analytics to Optimizely, there are a lot of tools out there that describe themselves as software for conversion rate optimization. While these options vary greatly in complexity, the best free tool is Google Analytics. If you’re not sure where to start, talk to a CRO expert at HALDA.
- Outline meaningful goals: Not every A/B test needs to be a home run. Depending on website traffic, you may be able to run multiple tests per month and still reach statistical significance. It’s more important that your testing is set up properly and that analytics events have been identified to correctly correlate with new applications.
- Ensure statistical significance: Statistical significance means that the sample size is large enough to be reflective of an entire population. Before you begin your first A/B test, you’ll need to determine how significant you want your results to be. The standard is 95% significant, sometimes referred to as confidence level. All this means is that there’s a 95% chance that the entire population of your website visitors will have the same outcome as the results of your a/b test. You can typically find your confidence level through the different tools you are using to measure your analytics like Optimizely or Google Analytics. To calculate your needed sample size, check out this calculator.
- The takeaway: A/B testing is one of the best ways to find what is most effective for your institution. Every school is different and just because one thing works for another university does not mean that it will work that well for you. A/B testing is a great way to refine your recruitment process, and HALDA is an easy way to start implementing this. We are always here to make your life easier.