I talk to a lot of people about the importance of SEO testing and how, for most large websites, organic search is their greatest opportunity for growth. But at the same time, it’s their least understood channel. When I ask customers and prospects why they are considering using SearchPilot as their SEO testing tool, three recurring themes always come up. Unsurprisingly, they are all things that help them get more winning SEO test results.
1. They want power
With SearchPilot, you can run pretty much any SEO test you can think of. You can use our SEO testing tool to test your on-page SEO, your internal linking, your meta data, and so on. Plus, our software works with Javascript frameworks like React and Next JS. This means more power, which means more freedom and fewer limitations when it comes to testing.
2. They want precision
The breakthroughs we’ve made in our statistical engine mean that SearchPilot’s analyses can detect smaller and more subtle uplifts with more confidence to help you find more winners.
3. And they want our professional services
Some of you might remember that SearchPilot grew out of an SEO agency called Distilled.
So although we now build and provide intelligent SEO testing software, we have professional services deep in our DNA. Our clients and prospects are often surprised by just how deep our team is prepared to go when it comes to analyses and supporting them on their testing journeys. In fact, I often believe that our professional services team has more experience running SEO tests than anyone else on the planet. Especially across the range and depth of industries and customer sites that we work with.
But you might be surprised to learn that our team uses additional technology behind the scenes. Not only do they have access to SearchPilot’s intelligent A/B testing platform that our customers use, but also some additional tools from our engineering and data science teams.
This helps them perform advanced tasks like lookalike bucketing and outlier detection and removal. It also gives them access to a variety of advanced statistics that helps them understand how well our systems model data, and enables them to debug any potential areas of confusion. The main tool I want to talk about is something we call The Long Room.
Introducing The Long Room
If we have any cricket fans among us, you’ll be familiar with The Long Room. For those who aren’t a fan of the sport, it’s a feature of the Lord’s Pavillion at Lord’s Cricket Ground, London. It’s the room players have to walk through it to reach the cricket ground. My co-founder and some of our early engineers are cricket fans, so they began naming our statistical models after cricketers and cricket terms. This information won’t make you any better at SEO, but I thought you might find it interesting!
Our version of The Long Room is a suite of powerful analysis tools that our professional services team can deploy on any individual test. This enables them to customize the statistical analysis for more detailed data. They can then push this data back into the client-facing dashboards for our customers to use and share with their wider teams.
Below, you’ll see I’ve run an advanced analysis in the tool. You can watch me discuss this in full here.
The charts below are similar to what you see in SearchPilot’s platform. This first one shows us the actual performance of variant pages compared to a forecast that we’ve run. And the second one drills down into the predicted uplift, along with the confidence interval that we have on that uplift.
The first difference you might notice, however, is that the charts feature additional confidence intervals. This is because I have requested extra ‘alphas’ in the settings. This means that we can see the core predicted uplift, or we can see the standard 95% confidence interval, or we can even overlay a confidence interval of 80 or 90%. This is useful for doing more advanced analyses, but it's still quite basic.
Let’s look at a more advanced tool. One of the more challenging pieces of analysis we're called upon to do is handling the impact of extreme outside events that throw off the test. This could be unexpected news coverage or an advertising campaign that runs during the test, only affects some website pages, and skews the results.
We call these outliers. Below, you can see that I’ve applied the ‘remove outliers’ step. This automatically discovers pages that don't perform like the rest of the site. And below that, you’ll see the tool identified one page with a huge spike that had very little traffic before the test started.
Removing it improves the model, as we can see when we compare the predicted uplift with a 95% confidence interval to the data we were seeing before.
These are our 'not so’ secret advanced tools that give our PS teams more insight into gathering and reporting on the SEO A/B tests we run for our customers. The advanced analysis power tool also gives our PS Team more insights into the outcomes of a test. Want to learn more? We’d love you to take SearchPilot for a test flight. Get in touch.
