Start here: how our SEO split tests work
If you aren't familiar with the fundamentals of how we run controlled SEO experiments that form the basis of all our case studies, then you might find it useful to start by reading the explanation at the end of this article before digesting the details of the case study below. If you'd like to get a new case study by email every two weeks, just enter your email address here.
This week, we asked our Twitter/X and LinkedIn followers what they thought happened to the organic traffic of pages for an e-commerce customer when we updated their product listing page to show content that had previously been hidden on mobile devices to now be visible in the mobile view.
This was what they thought:
The majority of voters thought this would have a positive impact, with just 13% thinking there would be no detectable impact. In this case, the majority of our followers were correct!
The Case Study
When designers choose which page elements to show on mobile view, it’s tempting to cut out text like content introductions to prioritize other page elements. This design is regularly chosen to optimize the limited space above the fold. However, since Google now prefers mobile-first indexing, choosing to hide introductory content on mobile can potentially make a big difference in what Google sees as the most important elements of a page.
One of our ecommerce customers decided to test whether there would be an SEO benefit to showing a content introduction on product listing pages that had previously been hidden on mobile devices. We hypothesized that showing this content on mobile would help Google understand the most important and relevant parts of their product listing pages, allowing these pages to rank higher for relevant keywords. Furthermore, we hypothesized that even though this change was only being made to mobile view, Google’s mobile-first indexing could help the pages result in an organic traffic uplift across all device types.
Results
This test resulted in a positive impact at an 80% confidence to organic traffic on mobile devices, with an estimated 7.5% increase in organic traffic.
Mobile device test results
But what about our hypothesis that a change to the mobile site would impact traffic on other devices as well? Across all devices, we still saw a positive impact at an 80% confidence, with an estimated 7.4% increase in organic traffic.
All device test results
Even though we only made the change to the mobile version of the site, the impact on rankings for these pages was seen across all devices. As Google continues to prioritize the quality of mobile users’ experiences with mobile-first indexing, we believe that SEO improvements on mobile view can positively affect traffic across all device types.
Our customer chose to have us implement this change using SearchPilot’s meta-CMS rule until their development team could implement it in production. This process enables us to implement prompt changes based on positive test results while allowing our clients time to develop a permanent solution.
We believe that these positive results were due to improved rankings for relevant search terms. We've seen in previous tests that the content of introductory text can be optimized to impact rankings. In this test, since Google saw introductory content that was relevant to these pages placed close to the top of the page on mobile view, it understood that this was important content that should inform its understanding of these pages’ purpose and help accurately answer users’ search queries. It’s also possible that Google saw this introductory content as another source of content it could pull to display in the SERPs. This therefore could have helped improve click-through rates, allowing it to show users content that closely matched their search queries.
To receive more insights from our testing sign up to our case study mailing list, and please feel free to get in touch if you want to learn more about this test or about our split testing platform more generally.
How our SEO split tests work
The most important thing to know is that our case studies are based on controlled experiments with control and variant pages:
- By detecting changes in performance of the variant pages compared to the control, we know that the measured effect was not caused by seasonality, sitewide changes, Google algorithm updates, competitor changes, or any other external impact.
- The statistical analysis compares the actual outcome to a forecast, and comes with a confidence interval so we know how certain we are the effect is real.
- We measure the impact on organic traffic in order to capture changes to rankings and/or changes to clickthrough rate (more here).
Read more about how SEO A/B testing works or get a demo of the SearchPilot platform.