When online shoppers make purchasing decisions—especially costly items—they typically follow a process that marketers call “the purchasing funnel.” As shoppers move through this funnel, their search behavior evolves. Early in the process, shoppers may use search engines to compare brands or determine which type of product suits their needs. Once narrowing down their choices and moving further down the funnel, their searches become more specific to certain models or features.
The Case Study
One of SearchPilot’s ecommerce customers sells products that require careful consideration before purchase, such as vehicles. To help those at different stages of the funnel, the customers' pages are structured to inform shoppers by adding abundant editorial content and common queries to help guide them toward a decision.
What was changed
As part of a broader project, our customer was creating a new informational page type, similar to product listing pages (PLPs) based on model or brand. However, they were concerned that these new pages would conflict with and cannibalize traffic from a different set of transactional pages on the site, which had been designed to be at the bottom of the funnel.
This existing transactional page type, which is a subset of the PLPs, included a reviews carousel at the bottom of the page, showcasing videos of experts sharing insights about the products based on its brand. These video reviews were YouTube embeds, a feature that can slow down page load, especially when multiple videos are loaded in the same element carousel.
To see the potential impact of removing this carousel, the customer ran a test that removed the expert video reviews embedded within an iframe on a set of pages that listed products by brand. They hypothesized that this change would negatively impact search performance; their goal was to measure how much organic traffic they might lose by doing so.
First Result
The results of this test were the opposite of what the customer was expecting. We saw a statistically significant positive impact at the 95% confidence interval by removing these video reviews, with a +4.1% most likely uplift in organic traffic.
Since these results were surprising, they decided to rerun this test on another subset of PLPs designed for users at a similar point in the funnel. This second iteration was run on pages that listed products by class rather than brand and included expert video reviews about various vehicles.
Second Result
Our model found that removing the expert video reviews from the class pages had no statistically significant impact on organic traffic, and the test was inconclusive.
The number of product listings on each page type may have influenced the difference in both test results. The class-based pages featured a broader selection of products, giving users more options to explore but also requiring more images and content to load; by contrast, the brand-based pages had a more limited product selection. Removing slow-loading elements like the review carousel with its embedded YouTube videos on these pages may have made a considerable difference in its’ page performance.
Another possible benefit was that removing the video review carousel significantly reduced layout shifting on page load. This improvement may have enhanced the page’s Core Web Vitals, potentially leading to a better user experience.
Given that these expert video reviews did not directly contribute to organic traffic, the customer decided that the carousel could safely be removed from these pages. While expert editorial content in the form of videos and reviews can be helpful for users, maintaining this content also requires additional infrastructure support from a site’s team. A test like this can save company resources by reassuring that removing a site element that has become expensive or time-consuming to maintain is safe.
When removing expert reviews or a similar video carousel, will you see a similar result? That depends on various factors unique to your website, which is why we recommend SEO testing whenever an enterprise-level site gets ready to deploy or remove a page element on a large group of pages. For even more information on the impact of adding or removing a page element, our full funnel testing methodology combines SEO and CRO insights to get the complete picture of how a change impacts your user base. Our case study on adding confidence messaging is an excellent example of how full funnel testing can give you a more complete picture of how a significant change can impact your site’s performance.
Want to know more about SEO testing at the enterprise level? Read more of our SEO case studies or get in touch with our team to find out if SEO testing is the right move for your website.
