Start here: how our SEO split tests work
If you aren't familiar with the fundamentals of how we run controlled SEO experiments that form the basis of all our case studies, then you might find it useful to start by reading the explanation at the end of this article before digesting the details of the case study below. If you'd like to get a new case study by email every two weeks, just enter your email address here.
In this week’s #SPQuiz, we asked our Twitter followers what they thought the outcome in organic traffic was when we added a new keyword to the H1 of a customer’s product pages.
Here’s what people thought:
The majority of our followers believe adding additional keywords to the H1 would positively impact traffic. In contrast, almost 30% and 7% believe this change would either not have a noticeable impact or have a negative impact on organic traffic, respectively.
In this case, the majority were correct…sort of - read the full case study below!
The Case Study
The H1 is one of the oldest known ways to improve relevancy as H1s are used to highlight the page’s main topic, contributing to the overall user experience. While the H1 has been standard in SEO best practice for years, its significance as a ranking factor has often been questioned and debated.
In early 2020, Google’s John Mueller took time to explain Google uses headings to understand page content better, and that this can result in ranking improvements, or in ranking for new searches.
We decided to experiment with the H1 of a customer’s website to see if it would allow Google to improve our rankings for a broader set of keywords. We ran this experiment with an ecommerce company that offers customizable products by adding the keyword “Custom” to the front of the H1 of the product pages. Note: “custom” already appeared elsewhere on the page, in the title, and in other headings.
In the image above, you can see the comparison between our example pages. The variant page now has an additional keyword, “Custom,” inserted in the front of the H1 to define our customers’ products and services further.
So, how did this test perform?
The chart below shows the impact of this test on organic sessions:
Our best estimate of the impact of this change was that it was positive, resulting in an 8% uplift to organic traffic, but it did not reach the threshold of being positive with standard 95% confidence.
Ultimately at SearchPilot, we understand that we’re doing business, not science; given the fact that this change was easy to implement, based on a strong hypothesis, and more likely positive than not, we made the decision to still deploy this change as a way to get any incremental gains on those pages that did exist.
If you want to learn more about that reasoning and how we make decisions on inconclusive test results, check out our blog post, “How to make sense of SEO A/B tests,” for additional reading.
Running this test demonstrated the influence the H1 can still have on your page when adding competitive keywords to it, such as “Custom,” despite the H1 having less significance than something like a meta title. It helps Google better understand the content, and this can impact organic traffic positively because Google can use these keywords to rank pages better to serve user search queries.
To receive more insights from our testing, sign up for our case study mailing list, and please feel free to get in touch if you want to learn more about this test or our split-testing platform more generally.
How our SEO split tests work
The most important thing to know is that our case studies are based on controlled experiments with control and variant pages:
- By detecting changes in performance of the variant pages compared to the control, we know that the measured effect was not caused by seasonality, sitewide changes, Google algorithm updates, competitor changes, or any other external impact.
- The statistical analysis compares the actual outcome to a forecast, and comes with a confidence interval so we know how certain we are the effect is real.
- We measure the impact on organic traffic in order to capture changes to rankings and/or changes to clickthrough rate (more here).
Read more about how SEO A/B testing works or get a demo of the SearchPilot platform.