Start here: how our SEO split tests work
If you aren't familiar with the fundamentals of how we run controlled SEO experiments that form the basis of all our case studies, then you might find it useful to start by reading the explanation at the end of this article before digesting the details of the case study below. If you'd like to get a new case study by email every two weeks, just enter your email address here.
In the latest edition of #SPQuiz, we asked our followers what they thought the SEO impact was of adding “with video” to a media site’s title tags.
Over half thought this addition would have a positive impact on organic traffic, you may be surprised to find out that this change was negative! Read on for the full case study:
The Case Study
Having video content as a complement to article copy can be a great way to engage users and enhance their overall experience, particularly in industries where video content helps illustrate things like “how-to”s.
One of our customers, a media brand, wanted to advertise this unique selling point in the search results by including in the title tag that the article had an accompanying video. We hypothesized that including a reference in the title to the fact that the article had both written and video content would make users more inclined to click through.
We ran two iterations of this title test, one adding “(video)” and the next adding “(with video).”
Here is how the first test, adding “(video),” looked in the search results:
Control | Variant |
---|---|
Control |
---|
Variant |
The result of this test was significantly negative:
We hypothesized that perhaps users read “(video)” as a sign that the page was solely video content as opposed to a contextual video within other written copy on the page.
With this in mind, we iterated upon this test by adding “with video” to the title tag instead. In this second test, our hypothesis was that indicating that a video is included within the content and not the sole form of medium included on the page could encourage users to click-through more often.
Here is how the second test appeared in the search results:
Control | Variant |
---|---|
Control |
---|
Variant |
The results of this second variation on the video title…
… had a significantly negative impact on organic traffic as well!
Indicating that the articles have an accompanying video directly in the search results was not impactful in the way that we anticipated. This test is yet another example where testing a change we had a logical basis for led to a surprising result.
In this case, it’s possible that adding length to the middle of the title tag truncated the brand name or other more relevant keywords, which could negatively impact CTR. It could also be that users landing from organic search for this particular industry place less value on video content than we thought, which may mean that seeing “video” in the title (even in the second iteration) deterred them from clicking through.
Test results like this can reveal gaps we may have otherwise missed and give hints as to what users are most drawn to. In this case it raises a bigger strategic question: does the outcome indicate that we may be better off making more written content instead of investing as much in video content? Of course, there are multiple factors at play that influence this kind of decision, but with the information gained by running these experiments, we are given more direction than we previously had!
How our SEO split tests work
The most important thing to know is that our case studies are based on controlled experiments with control and variant pages:
- By detecting changes in performance of the variant pages compared to the control, we know that the measured effect was not caused by seasonality, sitewide changes, Google algorithm updates, competitor changes, or any other external impact.
- The statistical analysis compares the actual outcome to a forecast, and comes with a confidence interval so we know how certain we are the effect is real.
- We measure the impact on organic traffic in order to capture changes to rankings and/or changes to clickthrough rate (more here).
Read more about how SEO A/B testing works or get a demo of the SearchPilot platform.