As with most experiments, SEO tests can go one of two ways. Either you reap the rewards and increase your organic growth, or you see minimal or negative impacts to your traffic.
It’s a volatile process and the risk factor can feel monumental, especially if your tests are driven by guesswork. This challenge is only exacerbated when you consider the resource demand and cost of more technical SEO tests, such as internal linking experiments.
That being said, if you play it safe and make no changes at all, your traffic can still suffer. This can be a result of factors such as new Google updates and competitor site changes.
If you accrue $2 billion in revenue from organic channels, a one percent drop in traffic would account for $20 million in losses.
So, how can you execute SEO tests that increase confidence and organic growth?
In the following eight steps, we’ll explain the scientific method for conducting winning SEO tests.
8 steps to increase traffic with confidence
1. Determine whether SEO testing is right for you
Hold up! We don’t want to waste your time researching SEO testing if it’s not suitable for you.
To put it simply, websites with a large number of pages which use the same template are the perfect fit for these experiments. It’s even better if pages are categorized so that you have levels of pages with similar templates. For example, imagine flight pages on a travel website. The ‘New York flights’ and ‘London flights’ pages will structurally follow the same content layout and format.
2. Choose the right tool
Let’s start by explaining which tools you shouldn’t use.
You may already use a host of SEO platforms, such as Ahrefs and Google Search Console, to inform your keyword strategies. While these tools are useful, you cannot use them for SEO testing.
This is because:
- It’s not possible to track every keyword your site might rank for. Not to mention, some keyword tools may not even have data on certain keywords.
- Almost half of Google Search Console clicks you receive go to hidden terms.
On top of this, you don’t want to opt for a testing tool that delivers client-side changes. These platforms are fine for user testing, but not for testing search performance. Why? Because you cannot guarantee which version of a page Googlebot crawls. With client-side changes, you often see the old version of a page first. If a Googlebot indexes this page instead of your updated page, it’ll invalidate your entire SEO test.
Without concrete data, you have no way of determining whether an SEO test was a success or failure. And, with around 80 percent of SEO experiments having a neutral or negative impact, this can prove disastrous.
All this being said, you need a tool that rolls out server-side changes and provides concrete impact data. That way, you can confidently determine the causal relationship between SEO tests and your positive (or negative results).
This is exactly how SearchPilot functions. We enable you to test changes, measure their impact, and ensure you never accidentally roll out negative changes.
You can learn more about our server-side SEO split testing here.
3. Determine test ideas
Once you choose the right SEO testing tool, it’s time to begin experimenting. To get started, you’ll want to brainstorm test ideas.
You can generate relevant ideas through your typical SEO recommendations approach, such as conducting site audits. For test ideas specifically, we like:
- Considering the SEO justification. You need to determine whether you can rank and, if so, whether you shouldrank. Ask yourself — is there any reason why Google can’t access your content? And, even if it is accessible, is it good enough that you deserve to rank?
- Comparing your website page templates with your competitors’ templates. Using a tool like Figma, create a visual board of competitor websites (mobile and desktop) and make a comparison table of template features. To make sure you’re looking at the same page as Google, use a structured data tool to search the HTML. We explain this in more detail in our webinar.
4. Create hypotheses
Any test you run is a combination of these three levers:
- Improving existing rankings.
- Ranking for more keywords.
- Improving organic click-through rates (CTRs) independent of improving rankings.
A test idea on its own is not a hypothesis. Like all scientific experiments, you need to follow a more considered approach. We recommend following Conversion.com’s hypothesis framework:
For a more in-depth look at how to create strong SEO hypotheses, we’d also suggest reading our dedicated blog post on the subject.
5. Prioritize your SEO experiments
Once you have a list of ideas and solid hypotheses for each test, you should then prioritize them based on effort and reward. Part of this involves weighing up the anticipated cost of a test if you choose to deploy it later on. Budgets will vary from business to business, so the scale and pace of changes you can deploy will differ.
6. Choose and split your pages
Let’s say your test involves moving content higher up on a page template. You hypothesize that this will increase your keyword relevancy and, thus, your ranking.
To test this hypothesis, you’ll need to split your pages into control and variant buckets. The control pages will stay the same, the variant pages will feature your changes.
When selecting these pages, choose:
- Hundreds (or, ideally, thousands) of pages to place into the buckets. The more pages you have, the better the data.
- Pages that have similar levels of traffic to ensure fairness.
- Statistically similar pages. For example, if you’re a travel website advertising beach holidays and ski destinations, divide these up between the buckets. Your ski destinations may receive higher traffic hits during the ski season, which can skew your testing data.
Tools like SearchPilot can do this bucketing for you automatically.
7. Make the desired change
If you’re using SearchPilot, deploying changes to your variant pages is a very straightforward process. Simply:
- Create a site section corresponding to the group of pages with the relevant template.
- Implement the change directly into the SearchPilot platform. This change updates dynamically on your live website.
- Measure your test data in the test dashboard (more on this in a moment), and monitor the percentage of test pages Googlebot crawls.
8. Detect statistical significance
Of course, to ensure you receive accurate traffic data, you need to create models for determining statistical significance.
To start, create two models for your control and variant pages, respectively, using historical organic traffic. These models should be a forecast of what you think should happen to the control and variant pages if you made no changes.
From here on out, you can measure your actual traffic day-by-day against your forecast expectations. Differences between control performance and forecast can be used to recalibrate the forecast for variant pages by capturing the effect of seasonality, Google updates, competitor activities etc.
Ultimately, if traffic to your variant pages goes up compared to the forecast, this indicates a positive result. You can then implement this change and — you guessed it — reap the rewards of a healthy traffic increase.
Conduct SEO experiments the scientific way
“Sometimes customers’ biggest wins come from never deploying losers rather than from only deploying winners” — Will Critchlow, SearchPilot CEO
A boost in organic traffic can have a direct, positive impact on your bottom line. On the flip side, a decline in traffic can cost you dearly. Blind SEO changes to your website can cause either scenario.
To ensure you implement winning changes (and not losing ones), you need to conduct data-driven SEO experiments. By testing hypotheses and receiving concrete causal data, you can better understand which changes to implement and which to avoid.
