Will adding AI-generated content to pages improve organic traffic?

Start here: how our SEO split tests work

If you aren't familiar with the fundamentals of how we run controlled SEO experiments that form the basis of all our case studies, then you might find it useful to start by reading the explanation at the end of this article before digesting the details of the case study below. If you'd like to get a new case study by email every two weeks, just enter your email address here.

In this week’s #SPQuiz, we test the addition of AI-generated content to pages to see if we can increase organic traffic. The customer is in the travel industry, what do you think the impact on their organic traffic was?

Twitter:

The Case Study

AI-generated content is quickly becoming a trending subject in the SEO-world. It possesses the ability to present itself in a conversational manner, making it more sympathetic with user queries and long-tail keywords. In the past, content often used to be straightforward and informative, but failed to precisely cater to user queries.

In an increasingly digital world, SEO plays a vital role in ensuring visibility and organic traffic for businesses. Content creation stands at the core of effective SEO strategies. Our customer in the travel industry tested the utilization of AI to rewrite informational text and demonstrate how AI-driven content creation can contribute to increased organic traffic.

During this test, our customer used artificial intelligence to create webpage content for two major travel markets: the United States and Australia. The experiment was conducted in both countries. Although the new content resulted in a remarkable 13% increase in the US market, we saw an inconclusive result on the Australia site, with a range of possible impacts between +12% and -17% in organic sessions. This highlights the crucial significance of testing. What works in one market might not work in another.

United States

US-market

Predicted uplift: 12.6% (est. 12,900 monthly organic sessions)
We are 95% confident that the monthly increase in organic sessions is between:
- Top: +26,700
- Bottom: +540

Australia

AUS-market

Predicted uplift: -3.3% (est -1,100 monthly organic sessions)
We are 95% confident that the monthly increase in organic sessions is between:
- Top: +4,000
- Bottom: -5,600

Since this test had different results based on the market, we recommended deploying it for the US pages, and taking learnings from the test on the Australian market. It could be a factor of needing to simply rewrite the Australian content using a different AI prompt, and iterating on what we tried previously, or running the test on a larger site section in order to increase the likelihood of a statistically significant result. We know that fresh, unique content is a positive in the eyes of Google, it’s just a matter of tweaking our approach enough to curtail it to the target audience.

If you are using AI-generated content and want to test its impact, click here to set up a conversation.

How our SEO split tests work

The most important thing to know is that our case studies are based on controlled experiments with control and variant pages:

By detecting changes in performance of the variant pages compared to the control, we know that the measured effect was not caused by seasonality, sitewide changes, Google algorithm updates, competitor changes, or any other external impact.
The statistical analysis compares the actual outcome to a forecast, and comes with a confidence interval so we know how certain we are the effect is real.
We measure the impact on organic traffic in order to capture changes to rankings and/or changes to clickthrough rate (more here).