For this week’s #SPQuiz, we asked our Twitter followers what they thought the impact would be when a travel website moved a deals block above their informational content. Here’s what they thought:
Over half thought that this would have a positive impact on organic traffic, the rest split evenly between negative and no detectable impact. This change actually had a negative impact on organic traffic! Read the case study below to learn more.
The Case Study
Have you ever launched a new page template that didn’t perform as well as you’d hoped? Sometimes this is inevitable, but with A/B testing we can at least measure the impact and garner some clues as to why the change might not be having the intended effect.
In this week’s case study, a travel customer had rolled out a new page template for their package holiday booking pages. They hadn’t been able to test it before rollout, and it wasn’t performing as well as they had hoped. The new page template had various components, like a block of travel deals, FAQs, content about the destination, and links to nearby destinations.
To explore what changes to the new template might improve organic traffic, they wanted to test changing the order of components on the page. They tested swapping the placement of the deals block with the block of copy about the destination.
In a previous test on a different page type, they added a similar deals block and saw a positive impact. So they hypothesised that if they emphasised the deal block more on this template by moving it further up the page, they’d see an uplift.
They hypothesised that having the offers higher on the page would make the page more closely match users’ intent. Alternatively, they were concerned that having the offers higher on the page could feel overly salesy which could devalue the fact that it was higher up on the page at all. They also considered that this change could devalue the “about destination” content, as it was now pushed further down the page.
This was the impact on organic traffic:
This test had a negative impact on organic traffic with an estimated -10% loss in organic sessions. Although it was not statistically significant at the 95% confidence interval, it was negative at a lower confidence level. You can read more about how we make decisions about tests that fail to reach statistical significance here.
Our best guess as to why this test was negative is that moving the informational content down the page decreased the importance of it. This could have led to an impact on rankings, which thus negatively impacted organic traffic.
Rolling out new layouts or page templates can have impacts on SEO that may be surprising. In this case, we only tested the SEO impact, but testing (if you can) for SEO, CRO or both can help you dissect which changes will have the biggest payoff.
