Start here: how our SEO split tests work
If you aren't familiar with the fundamentals of how we run controlled SEO experiments that form the basis of all our case studies, then you might find it useful to start by reading the explanation at the end of this article before digesting the details of the case study below. If you'd like to get a new case study by email every two weeks, just enter your email address here.
Earlier this week we asked our Twitter followers what they thought happened to organic traffic when an eCommerce customer tested fixing their breadcrumb markup so that the breadcrumbs would show up in the search results. Here are the results of the poll:
While majority of those who voted think this change had either a positive or no detectable impact on organic traffic, the result of this test was actually negative! Surprised? Read on for the full case study.
The Case Study
Breadcrumbs can be a powerful tool to improve internal linking and can also show in the search results if Google chooses to display them in place of the full URL. There are two main ways this can happen: 1. If you mark up the breadcrumbs using structured data or 2. If your URL path structure is easily understood, Google may automatically display it as breadcrumbs.
Not only do breadcrumbs help Google better understand your site structure by defining a hierarchy between specific pages, but if they appear in the search results, they also add the potential to drive additional clicks to any level of the hierarchy.
It used to be the case that they could only appear in the search results if you had the structured markup in place. This made them easier to test, and so we have some insights into their benefits. An eCommerce customer had breadcrumbs on their product and category pages, but an error in the structured markup prevented them from appearing in the search snippets at the time. They tested fixing their breadcrumb markup so that their breadcrumbs would show up in the search results as intended.
The hypothesis was that after fixing the breadcrumb markup the breadcrumbs would display in the search results and drive more clicks to the pages. Additionally, correcting any markup errors may help Google better understand the site’s structure and how product pages relate to their parent category pages and vice versa. This could result in improved rankings for the product and / or category pages.
Here’s how the control and variant pages looked in the search results page.
Control | Variant |
---|---|
Control |
---|
Variant |
We measured the impact on the product pages (e.g. /green-high-waisted-bikini-bottoms), subcategory pages (e.g. /clothing/swimwear) as well as the high-level category pages (e.g. /clothing):
The change’s impact on the high-level category pages was negative for SEO, with an estimated 7% loss in organic traffic.
The impact on the subcategory pages was likely also negative, with an estimated 6% loss in organic traffic.
On the product pages, the result was an estimated 12% loss in organic traffic.
The impact was negative on all levels - on the product pages, subcategories, and categories.
Before the fix, the full URL (e.g. sitename.com/clothing/swimwear) was showing in the search results. After the fix, that was replaced by “sitename > clothing”, without the most specific subcategory appearing.
To further illustrate this, below is what we thought our variant search result would look like versus how it actually showed up on the results page. We expected the subcategory breadcrumb “Swim & Beachwear” to appear in the search result after the top-level category “Clothing”, just how it was outlined in the markup. However, the variant page did not show the most specific level breadcrumb in the search results. Our theory is that this led to a less compelling snippet and fewer users clicking through.
How we hypothesized the variant would look:
How the variant actually appeared:
Breadcrumbs and its respective markup are often considered SEO “best practice”, however even if you’re able to get the rich snippet from breadcrumb markup, it may not improve your pages’ click through rate (as we saw in this test). Depending on the keywords used within the markup, the breadcrumbs displayed in the search results may be more or less compelling to users, directly influencing click through and your pages’ organic traffic.
How our SEO split tests work
The most important thing to know is that our case studies are based on controlled experiments with control and variant pages:
- By detecting changes in performance of the variant pages compared to the control, we know that the measured effect was not caused by seasonality, sitewide changes, Google algorithm updates, competitor changes, or any other external impact.
- The statistical analysis compares the actual outcome to a forecast, and comes with a confidence interval so we know how certain we are the effect is real.
- We measure the impact on organic traffic in order to capture changes to rankings and/or changes to clickthrough rate (more here).
Read more about how SEO A/B testing works or get a demo of the SearchPilot platform.