SEO A/B testing: The secret to increasing your organic search traffic

An employee analysing a dashboard at their desktop computer As a marketer, what does success mean to you?

We know, it’s a personal question. The answer will depend on your industry, your career path, and your longer-term ambitions.

But when it comes to the fundamentals of finding success in your role, the answer is quite simple. It’s all about results. You need to show an impact on fundamental business outcomes — revenue, profitability, and customer growth.

When it comes to moving the needle on these key metrics, search engine optimization (SEO) is one of your team’s most powerful tools. SEO can significantly increase the organic traffic to your website. And more traffic means more customers and more revenue. Simple, right?

But SEO isn’t a silver bullet for organic traffic. In reality, around 80 percent of website changes designed to improve organic performance either have no impact or actually decrease traffic. If you don’t have a reliable way to test and track your SEO efforts, you can end up expending a lot of effort for little return. And without being able to rule out a negative impact, you face the prospect of doing more harm than good.

So, how do you get the traffic-boosting benefits of SEO without the potential downsides?

The answer is SEO A/B testing.

SEO A/B testing provides the foundation for a data-driven, risk-free approach to SEO. It’ll help you outpace your competitors and deliver attributable results. And that’s a success by anyone’s definition.

Read on to find out how it works and the benefits it can bring to your business.

What is SEO A/B testing and why should be doing it?
The marketing benefits of SEO A/B testing
How does SEO A/B testing work?
3 examples of SEO A/B tests
Making SEO A/B testing a success
How SearchPilot can help

Get your PDF copy of this whitepaper here

What is SEO A/B testing and why should you be doing it?

The core challenge of SEO is that we don’t know exactly what Google wants.

Why do some pages rank higher than others for a specific search term? Google itself has confirmed (or denied) that certain factors play a role. But there is no comprehensive account of how its ranking algorithm works.

For this reason, SEO has traditionally involved making educated guesses about what search engines are looking for. You make changes to your site, cross your fingers, and keep an eye on your organic traffic.

Needless to say, for enterprises with large, mission-critical websites, this is not the optimal approach. You’ll find yourself making time-consuming and expensive changes to your site only to discover that they’ve had no discernible impact — or had a negative one.

Even worse, this approach makes it almost impossible to identify causal relationships between site changes and traffic uplifts. Your organic traffic is spiking — great! But did your SEO team’s changes cause it? Or was it some other factor, like seasonal shifts or Google updates? Without a way to tell for certain, you can’t draw any valid conclusions.

SEO A/B testing solves these issues. It allows you to test the impact of specific page changes before you deploy them permanently across your website. Once you’ve deployed them, you can directly track their impact on your organic traffic.

With SEO A/B testing, you have a measurable, risk-free way to deliver uplifts in your organic traffic — and a direct through-line to revenue, too.

features-seo-testing-retail

Looking for more information on whether SEO A/B testing is right for your business? These blog posts will help:

The marketing benefits of SEO A/B testing

SEO A/B testing doesn’t just have a tangible impact on your organic traffic. It also offers a range of other benefits for your marketing team, including:

Improved ROI. An SEO A/B testing program helps you identify and prioritize the most impactful site changes. Your team can implement the changes that deliver the most substantial uplifts in traffic while avoiding the ones that won’t. The result? An optimized return on your SEO budget.
Cost savings. With data on hand to guide your decisions, your engineering team won’t be wasting time on initiatives that don’t move the needle. Instead, they can focus on building out the most impactful changes, saving up to 80% of their effort while capturing all of the rewards.
Confidence. With every initiative tested in advance, any changes you deploy will be backed by data. You’ll be able to forecast percentage traffic uplifts based on your test data and track their impact in real-time.
Competitiveness. A comprehensive SEO testing program helps you deploy the most impactful changes quickly. You’ll be able to outpace your competitors in securing top rankings for key terms — and that means a larger share of search traffic.
User experience. Many of the tests that affect SEO will also have an impact on your user experience. Streamlining your search navigation or improving your page loading times aren’t just good for Google — your website visitors will like them, too.

Two employees discussing how SEO AB testing works

How does SEO A/B testing work?

Hopefully, we’ve convinced you of the value SEO A/B testing can bring to your business. But how does it actually work?

Part of the reason that SEO A/B testing has yet to be widely adopted is that it poses some unique difficulties. Most familiar forms of A/B testing involve comparing how users engage with two different versions of the same web page. But this doesn’t work for SEO. Search engines only see one version of each page. Trying to serve different versions of a page to users and to search engine crawlers is classed as cloaking, and Google really does not like this.

At SearchPilot, we’ve developed a platform that supports the only true method for SEO testing: splitting pages, not users. Here’s how it works:

First, you identify a group of similar pages on your website — pages that share the same template or the same basic features. Ideally, there will be lots of pages that fit the bill, such as destination-specific pages on a travel website.
Then, you implement the change you want to test on half of these pages. This is called the variant group. The rest of the pages stay the same — this is the control group. Unlike with CRO tests, these changes are implemented on the server side, rather than in a user’s browser.
Finally, you see what impact this has on the pages’ organic search performance. If the variant pages begin to perform better than expected compared to the control pages, you can be confident your change is delivering an uplift. If not, then you’re likely better off sticking with the existing page.

SEO A/B testing is particularly effective if your website has a large number of similar pages and receives high volumes of traffic. You’ll be able to test more pages and get a clearer picture of any impact. And the more traffic you receive, the greater the consequence of a percentage uplift.

For a deeper look into how SEO A/B testing works, check out the following resources:

3 examples of SEO A/B tests

So, that’s the theory behind SEO A/B testing — but what does it look like in practice? To give you a clearer picture, let’s look at three examples of tests SearchPilot has delivered across different verticals.

Retail: product description placement
Travel: pricing in page titles
Jobs listings: accurate vs approximate numbers

The following examples indicate just how complex the SEO impact of site changes can be. Even minor adjustments can lead to significant improvements — or disastrous declines.

SearchPilots examples of SEO AB tests (1)

Product description placement for retail

1. Retail: product description placement

Product descriptions are an essential feature of ecommerce sites — but where should you put them?

We experimented with the impact of placing product descriptions “above the fold” so that visitors would see them as soon as they landed on the page. By comparison with pages where the product description was featured lower down the page, we saw a 14% uplift in organic traffic.

This suggests that rolling out a redesign of your product pages could have a major impact.

More retail SEO tests:

2. Travel: pricing in page titles

For travel websites, featuring prices in your page titles is a common SEO strategy. But is it actually a good idea? And if you do include prices, should they be static or dynamic?

This is a case where SEO A/B testing reveals an unexpected outcome. We found that adding static pricing to the page titles of a travel rental site actually decreased organic traffic by 7%. However, when we retested with dynamic pricing, we found the opposite result: traffic increased by 10%.

This kind of nuance is only identifiable through SEO A/B testing. Without a way to effectively measure such a subtle difference, you have to rely on luck to help you land on the right approach.

More travel SEO tests:

3. Job listings: accurate vs approximate numbers

Using numbers in page titles is commonly assumed to help attract more users — but should you use the exact number or an approximation?

We commonly see sites using “10+” or “100s” in their titles, but we wanted to know if this was preferable to providing the exact figure. Our tests on a job listing website found that using an approximate number likely had a negative impact. While we weren’t able to reach a 95 percent confidence threshold, it was at least clear that making the change wouldn’t secure a benefit.

This is a handy reminder that SEO A/B testing isn’t just about positive results — it also helps you avoid wasting energy and resources on changes that don’t make a difference.

More listings SEO tests:

Making SEO A/B testing a success

So, you’ve seen what benefits SEO A/B testing can bring and the kinds of uplifts it can deliver. But how do you actually put it into practice?

There are a few things you and your team will need to do to effectively implement SEO A/B testing:

Communicate the benefits. You may see the value in SEO A/B testing, but you’ll need support to secure buy-in from other senior leaders. We can help you clarify and communicate the value of a structured SEO testing program. A stronger insight into the technical benefits will strengthen your overall case.
Measuring success. Your SEO team will be responsible for designing the tests and, most importantly, reporting on the outcomes. They will need to perform focused, high-value tests based on valid hypotheses. From this, you should provide specific, measurable outcomes to highlight the impact your new approach is having. Without a direct tie to revenue, you’ll struggle to prove the value of your efforts.
Choose the right tool. A DIY approach to SEO A/B testing is unlikely to deliver the results you (and senior leadership) are looking for. You need to be able to test large numbers of pages quickly and analyze the results with confidence. Trying to do this manually — or using an SEO tool that doesn’t support this functionality — will result in test results you can’t trust. You need a dedicated SEO testing tool that is scalable, secure, and easy to implement.

How SearchPilot can help

SEO A/B testing offers a bold new approach to SEO. It empowers you to deliver measurable uplifts to your organic traffic. It’s good for you, it’s good for your team, and it’s good for the business. So, why wait?

SearchPilot can help you deliver large-scale testing programs at pace, with a direct and attributable impact on your ROI. Our SEO A/B testing platform is user-friendly and easy to implement, integrating directly with your existing tech stack.

No matter the scale of your site or the volume of traffic you handle, we can deliver risk-free testing that will help you achieve your strategic goals. Book a demo with one of our experts for an in-depth conversation about how it can help you.

Share this post