Like any good creative output, the first version of your landing page will always have room for improvement. This is the first impression many users will have of your business, so you want to get it right, but it’s also important not to delay launching until you think it’s perfect.
So what options do you have when you need to get your landing page live, but you also want to make sure it’s pulling its weight marketing your business? Enter: A/B testing.
What is A/B Testing?
Also known as split testing, an A/B test is a type of marketing experiment where two versions of the same output — a landing page, an interior website page, even an app — are run at the same time to a similar audience. The goal is to determine which minor differences in the outputs are most influential and, ultimately, which version generates the highest conversion rate.
For example, on a landing page, you might want to test the color of a call to action button or a different attention-grabbing headline. You’re not sure which option will be most enticing to an audience, so you set up an A/B test to experiment. Only one element on the page will be different, to ensure that this aspect is what’s driving the higher conversion rate and nothing else on the page. You’ll then run the test to see which variation is more successful.
How Does A/B Testing Work?
You can run two variations of the same landing page, Version A (the control) and Version B (the test), at the same time to different segments of your target audience. Visitors to your site will be randomly assigned to one of the two versions — this is important for ensuring that the test is unbiased and as accurate as possible.
From there, once enough traffic data has been collected, you can analyze the results to determine which option provided the highest conversion rate, Version A or Version B. Whichever option has the highest result can then be implemented long-term as the main landing page, as this provides a higher level of confidence that future success will be seen versus implementing the poorer performing version.
Why Are A/B Tests Important?
The ultimate goal of an A/B test is to remove the guesswork in marketing to your audience. Rather than making assumptions about which headline users prefer or if a call to action should be clearer, the test can provide hard data on which option an audience likes best.
By continually running A/B tests and making incremental changes to your site, you can refine your landing pages over time. This will significantly improve the user experience of interacting with your site and should help to increase sales moving forward. Even negative results are worthwhile, as these will guide you in what not to do for your next test and can save countless hours of testing on aspects that you already know don’t perform well.
Another benefit of A/B testing is that it’s a relatively low-risk way to determine what’s working well, with the potential for high rewards. Since you’re only making small changes each time, it’s unlikely that conversion rates on either version will negatively impact the business’ bottom line, but if you find that one change can generate a higher positive result, that’s a good thing!
What Can You Test on Your Landing Page?
Headings and Copy
Both main headings and subheadings can be tested as part of an A/B test. One of the best ways to do this is to use different wording or tone when changing a key heading on the page. Even altering the length of the heading can be an impactful area to test.
Try to emphasize different features or benefits of your product or service when experimenting with headings. Question-based headings versus statement-based ones can also be useful for testing to see which version your audience responds best to.
On-page copy can also be interesting to test. Try not to change too much at once, though: A few sentences should be plenty. You can also run copy tests for tone, as with headings, or shorter sections versus longer paragraphs.
Calls to Action (CTAs)
You should run both the text and visuals of a CTA through A/B testing. The colors, placement, and even sizes of buttons encouraging users to take a next step should all be run through various tests. The different wording of CTAs is also important, e.g. “Buy Now” versus “Get Yours Today” could generate a clear winner that demonstrates what your audience prefers.
Visuals
If you’re using images or videos on your landing page, you should test both the content itself and its placement on the page. Experiment with the use of white space around images and videos to see if this helps them stand out to your audience.
Static images versus video content can also be interesting to test, particularly if you’re trying to make a case for investing more in video content creation as part of the ongoing marketing budget.
Page Layout or Design
You can also test the actual formatting and layout of the landing page. Decide where you want CTAs to go throughout the page, where information needs to live, if it should be split into different sections, and how long the page is.
You can incorporate heat-mapping software into the experiment to determine metrics like time on page, how far down the page users are scrolling, where they’re clicking, and what other actions they’re taking.
Deciding Which Variables to Test
Analyze Current Performance
Once you’ve determined what outcome you want from your A/B test, you should review the current performance of any landing pages or key site pages you have. This will give you a good baseline for key metrics like click through rate (CTR), bounce rate, and on-site engagement.
Prioritize the Highest-Impact Areas
The features you change in your test should always reflect your goal. If you’re looking to improve user experience, you’ll want to focus on areas that have a direct impact, such as navigation, page length, and messaging clarity. For potential revenue impacts, consider the areas around pricing, discounts, or trust signals like testimonials.
Review Customer Feedback
Look at what your customers are saying about your website already if you need additional ideas for what to test. If they mention confusion in navigating around the site, consider making the simplification of your nav the main focus of your A/B test. If they point out other frustrations in surveys or reviews, take this as inspiration for which variables to change.
How to Run an A/B Test to Optimize Your Landing Page for Conversions
Step 1: Identify Your Experiment Goal
Always be clear about the intended outcome of your experiment before you get started. Define what you want to achieve from your A/B test upfront, as this will determine both your hypothesis and which variants you want to change.
Step 2: Create a Hypothesis
A hypothesis is a clear, predictive statement about what you expect to happen in the test. It doesn’t need to be as specific as “changing the color of the CTA button from green to pink will increase conversions by 10%” — although it can be if you’re confident! You can simply state which option you think will generate a conversion increase, without a fixed number attached.
Step 3: Create Your Variant Page
Develop the new element for your Version B page of whatever variant you’re testing. Remember to only test one element at a time to generate a more accurate result and truer reflection of what the audience prefers. The more variants you change at once, the more difficult it is to determine which is causing an improved result.
Step 4: Run Your Test
When the variants are ready, you can launch and run your test. Be sure to run the test for long enough to determine statistical significance. What this means to you and your business will vary depending on how much web traffic is normal for you.
Step 5: Analyze the Results
Once you have a large enough sample, you can end the test and determine which version performed better. Some of the metrics you might want to compare are:
- Click-through rate, especially if CTA buttons were being tested.
- Engagement rate.
- Time on page.
- Time on site.
- Conversions.
- Bounce rate.
Step 6: Launch the Winning Page
When you know that you have a clear winner from the comparison between Version A and Version B, you can roll out the better-performing landing page as your main page for the service or product. From there, you can also run additional A/B tests.
Examples of Great Landing Page A/B Tests
Clarins
When launching new products, skincare and cosmetics brand Clarins often runs A/B tests to determine what their audience prefers, such as how explanatory text around images affects sales. They found that when images included copy, more users added the product to their cart.
Electronic Arts (EA)
When rolling out the SimCity5 video game, EA tested a discount offer on their landing page as part of an A/B test. While it may be tempting to assume that a discount for future games would be more enticing to audiences, that wasn’t the case. Per Unbounce, the non-discounted page had a 40% higher conversion rate — users simply wanted to play SimCity5 and weren’t interested in a discount.
Multivariate vs. A/B Testing: What’s the Difference?
While A/B testing typically only looks at testing a Version 1 versus Version 2 of a page, multivariate testing involves changing multiple elements on a page at the same time, during the same test. This can be beneficial if you’re looking to change different aspects, but they can be more difficult to review and analyze once the test ends. Testing too many options simultaneously can make pinpointing the deciding factor on results improvements more challenging.
What Not to Do When A/B Testing
Not Considering Past Results From Other A/B Tests
If you’ve run A/B tests before, it’s vital that you take those results into consideration. If you end up changing the same variant in your next test, you’re wasting your time. Review what’s worked well before and keep those elements in both so that you’re not repeating the same test over again on different pages.
Changing Variables in the Middle of a Test
You should never update your test pages once they’ve started running. If you need to tweak an element, you should end the current test and start a new one. By changing variables in the middle of a test, your results won’t accurately reflect the original test and make it more difficult to determine which variant was better.
Not Running Tests for Long Enough
One of the biggest mistakes people make with A/B testing is not running their tests for long enough to get a good representation of what the audience thinks. Your sample size needs to be large enough for statistical significance, so all testing should be run for at least several weeks, if not longer.
Recommended Tools for A/B Testing
A Landing Page Builder
Using a tool like a landing page builder helps you get started much faster with your A/B tests, particularly if you don’t have a web design background. Tools like this make building a landing page from scratch quick and simple, while allowing you to duplicate existing creations and make the small changes you need to run an A/B test.
A Testing Platform
You’ll need to use a specific A/B testing platform or website builder that offers this option in order to run your test. These tools will gather data and allow you to host two versions of the live landing page, while automatically splitting your users into random groups for the test. For more in-depth options, platforms like Realize offer AI-assisted A/B testing, allowing you to optimize and make improvements in real time.
Realize's advanced AI capabilities enable precise targeting, engagement optimization, and budget simulation, maximizing advertiser ROI.
Analytics Tracking
Tools like Google Analytics are a great, free option for tracking the web traffic on your landing pages. Give each variant its own unique URL to track data more effectively and accurately, while attributing it to the relevant page.
Terms to Become Familiar With
Here are several key terms that you should be aware of when running A/B tests:
- Variant: This is the element of each version of your page that you’re testing, e.g. the CTA or main heading.
- Control:The original page is known as the control, as you’re not changing any element of it for the test. The control is the opposite of the variant, which is the page you are changing.
- Statistical significance: This refers to both the validity and reliability of the results you generate. You must run tests long enough to achieve this.
Key Takeaways
A/B testing is not a one-time tactic for determining marketing improvements. Instead, you should use this method continually to test different elements of a landing page and to refine your current design and copy. Not every test will demonstrate dramatic results, but small, incremental changes over time as a result of A/B testing can have a significant impact on business growth.
Frequently Asked Questions (FAQs)
How long should an A/B test run for?
Every business will need a different sample size to reach statistical significance. If you have high traffic, you could reach this in several days, while other businesses may need a month or longer.
What is a typical success rate for an A/B test?
For most A/B tests, you want to reach an average statistical significance of around 20-30%. If your success is more than 50%, you’ll want to re-run your test after making changes to your control page, as this would suggest a major difference in quality between your control and variant pages.
What is split testing on landing pages?
Split tests can often be used to describe A/B tests, but they can also be used for much larger changes that need more in-depth analysis of results.
