Should we worry about running out of test ideas?
This is a question that comes up reasonably often in discussions with prospective SearchPilot customers, and it’s one that I have answers to in three different lengths and complexities:
The one-word answer to “will we run out of test ideas?”
No
The tweet-length answer to “how can we be confident we won’t run out of test ideas?”
My best single-tweet-length evidence is that across the dozens and dozens of SEO testing programs I've been involved with, I've never seen one fail for lack of ideas.
— Will Critchlow (@willcritchlow) September 2, 2022
The post-length answer to “how can we make sure we don’t run out of test ideas?”
See below!
How can we make sure we don’t run out of test ideas?
On any website that drives enough revenue to build a strong business case for testing it tends to be easy to generate test ideas in my experience.
As a reminder: great sites for testing are those where percentage-point uplifts in organic search revenue are meaningful and where organic traffic is distributed across many pages in large site sections. For these kinds of sites, my experience is that reasonable assumptions about numbers of tests, win-rates, and the retention of uplifts over time lead to strong ROI and business cases for investing in testing.
Here are ten ways of generating more test ideas. I guarantee that if you work on a site like I described, and you run through all of these, you’ll have a backlog as long as your arm.
1. Check the old backlog of SEO tickets
Every team has somewhere that they keep the backlog. If you’re really lucky, you’ll have multiple places. Bonus points for having a “unified backlog” to be the source of truth across all the old backlogs, which just becomes yet another backlog. Point being, wherever you keep them, whether it’s Jira, Trello, Excel, or a stack of print outs in a desk draw, there will be unloved requests from months gone by that are currently doomed never to reach the front of the line.
These can be fertile ground for new test ideas, because they often contain plausible hypotheses that don’t come with a strong enough business case to make it easy to argue that they should push ahead of product enhancements.
2. Dig out old SEO audits
Sometimes the ideas don’t even make it to the engineering backlog. Sometimes they remain stuck on page 59 of a 200 page SEO audit. Most SEO agencies deliver audits - though the best know that you don’t win by delivering a bigger document - at Distilled we used to say “it’s not our job to deliver reports, it’s our job to effect change”. Most in-house SEO teams have multiple audits - at least one from before they arrived and one from their preferred agency. No matter how good the agency and the relationship, it’s rare for every good idea to have reached the engineering backlog, so it’s worth digging out the old audits to look for the ideas that have been difficult to prioritise.
3. Do a focused audit
The last thing you probably need is yet another full audit, but there are a few key areas of on-site setup and structure that are super amenable to testing and that often generate great test hypotheses, so do a mini audit focusing on:
- Internal anchor text and linking structure in scalable site sections (focus on links “across” and “up and down” the information architecture)
- Structured data on page templates with tons of pages
- On-page keyword targeting and meta information including titles
4. Run a crawl
Maybe this is 3.a but don’t just eyeball the scalable sections - run a crawl using your crawler of choice to focus on linking structure in particular.
5. Take a fresh look at content quality
Content quality is a hot topic (thanks helpful content update) but even without updates, we know that Google in particular has been pushing more and more in the direction of wanting to reward content quality and ignore low quality content. Meanwhile, most large sites have sections with content elements that no one can be honestly proud of. It’s not guaranteed that enhancing that content will generate an uplift, but it’s a solid place to look for hypotheses.
Some specific things to look out for:
- Content quality (especially on e.g. category pages) - see our content quality SEO test case study
- Content location on the page - see our SEO test results for HTML structure, page layout changes and position relative to the “fold”
- Display of content (e.g. tabs / accordions) - we have interesting SEO test results in both directions (tabs / accordions) in different circumstances so this is one hypothesis that definitely needs testing
6. Review how your site appears in the search results (SERPs)
Search snippets are too-often overlooked in SEO, as they can have huge impacts on clickthrough rate which can drive immense traffic swings even without ranking changes. Of course some elements like titles are even more powerful as they can affect both ranking and clickthrough rate.
Are you getting featured snippets? Rich snippets? Are your titles and meta descriptions getting overwritten?
All of these questions can point to test ideas and hypotheses.
7. Review your competitors’ websites and how they appear in the SERPs
Spend time on your competitors' websites. Look at how they appear in the SERPs. Take time to map out what elements any of them have that you don’t, or what’s more prominent on their sites than yours. Can you get keyword ideas from how any of them present themselves?
There are significant risks to copying competitors blindly - even if they outrank you for a specific keyword, there is in guarantee that specific differences are the reason, and they may not be taking a data-driven approach - but competitor research is a superb idea generation starting point. All the things you see any of your competitors doing can go in the hopper of test ideas to be prioritised and checked.
8. Do some keyword research
In my post entitled marginal losses: the hidden reason your SEO performance is lagging, I wrote:
For example: who among us has carried out keyword research into the different ways people search for key content across a site section, determined that there is a form of words that has a better combination of volume vs competitiveness and made a recommendation to update keyword targeting across that site section?
Everyone. Every single SEO has done this. And there’s a good chance you’ve made things worse at least some of the time.
The risks of recommendations based on keyword research alone don’t apply to test ideation though. Just like with competitor research mentioned above, insights from keyword research can be a great source of ideas for tests. This is especially true because they are super high variance - some of our biggest winners and our biggest losers have come from ideas originating in how the target market searches. This makes them a superb source of test hypotheses where variance is your friend as long as you roll back losers quickly.
9. Get some fresh ideas on the problem
Ask for ideas from your peers. Do a swap with an in-house friend at a non-competitive brand. Heck, if you’re in our target market, you might get some free ideas out of us if you get a demo, but don’t tell the team I said that.
I always recommend in-house SEOs find communities to join where they can meet other smart thinkers and bounce their ideas around. One of the benefits of working for an agency (or in a professional services team like ours) is the number of SEO experts you are around continually and the quality of the discussion in the internal slack channels. Even if you are lucky enough to work with a large in-house SEO team, everyone is focused on the one company by definition, and many in-house teams are small, so branching out and getting diverse ideas is critical.
10. Have a browse through our published test results and case studies for ideas
We publish a new case study and test result every two weeks and we have dozens and dozens of them in the archive. You can subscribe here to get them in your inbox every two weeks and be among the first to hear about each new test we publish.
11. Bonus: DM me
My DMs are open on twitter. If you work for a large site with tons of organic search traffic to high numbers of pages in a templated site section, then you can definitely nerd snipe me because I’m fascinated by big websites!
Continue the discussion on twitter
I published this conversation to twitter for discussion - join in here:
Something I hear reasonably often when we are discussing SEO testing programs with prospective @SearchPilot customers:
— Will Critchlow (@willcritchlow) September 2, 2022
"Should we worry about running out of SEO test ideas?"
The short answer is "no". Here's the longer answer: pic.twitter.com/sloGQv5Keo
