SEO Testing: What is it and how to run SEO tests?
I joined SEOTesting back in April 2023. Since then, we have seen remarkable growth in organic traffic:
The secret to this growth? Tons of content created, links built, and site improvements made. But there was one other technique we used.
SEO testing.
In this article, I’ll explain what SEO testing is, why you should perform it on your website, the different types of SEO tests, and how to analyze test results. I’ll also provide SEO test examples for you to run on your website.
Let’s get into it.
What is SEO testing?
SEO testing involves making a change to your website that might affect your SEO, and then measuring its impact to understand its relationship to the results.
When I think about it, my company has been doing “SEO testing” since SEO became a full-time role. We would make all kinds of changes websites, including:
- Updating page content
- Improving the meta information
- Adding images
- Building backlinks
We would also monitor something else, such as website rankings, to review the impact of that change. We might, for example, see that acquiring two new backlinks to a page improved its ranking for its target keyword by four places. This would be an example of a successful SEO test.
SEO testing looks a little different in 2024. These days, we have access to diverse, robust toolsets that allow us to track almost anything on our website, whether its traffic, user experience, rankings, or any metric an SEO team could be responsible for (in full or in part).
Here’s an example:
Our website has a blog post on creating monthly SEO reports.
According to SE Ranking, this blog post is currently in Position 6 in the USA for its target keyword.
We might, for instance, add internal links to this blog post to see if that improves its ranking for that page for a particular keyword.
We’d make the change, choose a period to monitor the page for, and then monitor it to see if anything has changed.
What makes SEO testing useful for websites?
I work for an SEO testing tool, so my opinion on its usefulness might be a bit biased. Still, I’ve been running regular SEO tests long before working for SEOTesting. SEO testing is something all SEO professionals do. We make a change, and then we monitor metrics to see what happens.
However, some sites will benefit more from SEO testing than others.
- Data-driven decision making
One of the main advantages of running regular SEO tests is the ability to make informed, data-driven decisions. Instead of relying solely on traditional SEO activities like acquiring backlinks, publishing content, or tweaking meta tags (although these are still useful), SEO testing allows you to experiment with changes, analyze the data, and learn from those outcomes.
Running regular SEO tests helps you continuously refine your approach, much like we’ve done on our website. It also leads to major improvements in organic traffic acquisition:
- Improving website performance
SEO testing doesn’t just help you increase your traffic. It’s also a great way to improve your website’s user experience, which is a vital ranking factor in Google’s algorithms. Monitoring your user activity with tools like GA4 helps you see how making adjustments to your site affects behavior. For example, GA4 can show you conversion rate and lead generation spikes.
Tweaking on-page elements like CTAs, content layout, and meta information can help your site attract more visitors to convert into leads or customers.
- Understanding what doesn’t work
It’s important to realize that not every test will succeed, which is ok. It’s part of the process.
Some changes may not result in the changes you hope for, but this is how SEO testing provides additional value. Embracing the learning process and basing decisions on real data keeps your ongoing SEO efforts precise, effective, and beneficial for your users.
Even when tests don’t work as expected, they still provide valuable insights. SEO testing helps you refine your strategies and focus on more meaningful performance indicators.
Types of SEO tests
There isn’t just “one” type of SEO test. There are three.
You can choose to conduct single-page tests, group tests, or even split tests. We’ll take you through each of them in the following section
Single-page tests
Single-page tests focus on optimizing and experimenting with the SEO elements of individual pages. This gives you an idea of how changes impact their standing in search. It tracks metrics like organic clicks, impressions, CTR, and the individual page’s average position.
Let’s say, for example, that a landing page for a product or service you offer isn’t performing well in search results. It would be beneficial to perform a single-page test so that you can experiment with things like:
- Title tags and meta descriptions: To improve click-through rates
- On-page optimizations: Improving the content on the page
- Changing header tags: For better structure and relevance
- Testing internal links to this page: Specifically from other important (and relevant) pages on your site
You could, for example, tweak the page’s meta description and check if this has any impact on its click-through rates over time. This is a prime example of a meaningful single-page SEO test.
Group tests
Like single-page SEO tests, group tests apply the same principles, but with one crucial difference. They involve making the exact change to a group of pages rather than just one page on your site.
Imagine you’re the team lead for the SEO department at an ecommerce site with multiple product pages. You notice that these pages are not ranking well. One possible response you could take is to group similar product pages (like all product pages focusing on men’s running shoes) and run an SEO test. Consider experimenting on:
- Uniform meta descriptions across all pages in the group.
- Schema markup, like adding product schema to all product pages.
- Changing the URL structure of this group for better URL optimization.
The test will then run across the group. Then you can use those results and apply what you have learned to other groups across the site.
Split tests
Before getting into this, there’s one crucial distinction you should be aware of. Split tests are sometimes referred to as A/B tests. This is not the case in SEO testing. A/B tests refer to sending users to two versions of the same page. This isn’t possible with SEO, but it is when conducting UX experiments. So, when we refer to split testing, we are not referring to traditional A/B testing.
When you conduct split tests, you divide your page variations into two groups, comparing the performance of one group to the other after making a change. This test type is valuable in helping you determine which page version performs better organically.
You might be considering changing your blog format to a completely different one. But before doing this, consider conducting a split test. Split your blog posts into two groups, make sure the two groups get approximately the same traffic and make changes to all pages in one group. This is your test group. Compare this group’s performance to your control group (the group of pages you did not change) to find out which version of your page performs better.
Once you have this information, you can either:
- Make the change to all your pages if the test group performs better.
- Revert the changes if the test group performs worse.
How to run SEO split tests
Given that split tests can be the most complicated to run, let’s go over the process in detail. Below are the four stages to running an SEO split test:
- Choosing a hypothesis.
- Choosing pages for your split test.
- Establishing a period for the test.
- Running the test.
Choosing a hypothesis to test
The first step is deciding on a hypothesis to test. This is where your data comes into play.
Start by identifying potential areas for improvement based on what your performance data tells you. Maybe your pages are getting lots of impressions but only a few clicks. This could mean your titles or meta descriptions must be more compelling, and tweaking them might improve your CTR. Alternatively, there may be a group of pages on your site that aren’t ranking as well as you’d like them to. Another dilemma is that the content is well-written and valuable, so it should perform well. You might hypothesize that improving internal links or adding schema markup will help.
In short, start by looking for promising, low-hanging fruit. Improve a specific metric like CTR, traffic, or the page’s ranking for target keywords. Use tools like Google Search Console to identify areas for improvement.
You will be left with is a 1-2 sentence hypothesis to test, something like:
Choosing pages to test
When you’ve settled on a hypothesis, the next step is choosing the right pages to test. A valid split test requires both a test group and a control group. These groups should include strikingly similar pages. These pages should follow the same template, have similar traffic levels, and, ideally, focus on similar topics or products.
For example, if you’re testing product description pages (PDPs) on an e-commerce site, all the test and control groups’ pages should be PDPs. Avoid mixing page types, like blog posts, with PDPs. This will skew your results. Your goal should be to isolate the variable you’re testing. This will ensure that any difference in performance is directly attributed to your changes.
If you are struggling with this, SEOTesting offers a tool that automatically chooses the best control and test group URLs:
Deciding on a time frame for the split test
The duration of your test depends on your traffic levels. Generally, you’ll want to run an SEO split test for at least 2–4 weeks. Lower-traffic sites may need more extended testing periods to get statistically significant results. Keep an eye on the data, but resist cutting the test short. Premature conclusions may give inaccurate insights. In short, patience is key.
Running your split test
The last step is to run the test. Once you’ve completed all of the steps above, formulated a hypothesis, established your control and test groups, and decided how long to run the test, it’s time to make the required changes to all pages included in your test group. Then you must begin monitoring your data.
This is the most time-consuming part of the process because it requires taking a manual approach.
For each day during the test, collect performance data for both your control and test URLs, including:
- Organic traffic
- Impressions
- CTR
- Average position
But you may need to collect other information if you are testing for improvements in other metrics, like engagement rate, for example. The ideal result will be a spreadsheet that looks something like this:
From here, you would analyze all of your raw data to determine the overall success of your split test.
To save time, you can also use SEOTesting. This will allow you to setup the test in 5 minutes, and make changes to your test group URLs (manually). From there, SEOTesting will do the work for you. It will also automatically all of the data for you and give you results that are much easier to read and understand. Let’s take the raw data above and put this into SEOTesting:
As you can see, it is a very successful split test.
SEO test examples
Now it’s time to put all of this information to practical use. Below are five examples of SEO tests that you can perform right now.
Meta title & meta description changes
It’s common among SEOs to update meta titles and meta descriptions, but tracking these changes is always worth it (use an SEO testing tool or Google Search Console). This ensures they have a positive impact.
Here’s an excellent example:
We’ve all written guides and put the year within the title and/or meta description, right? Google values relevancy, which is common practice for many of us. If this is the case for you, return to your older articles and update the year to the current year. You might be surprised at just how good the results are.
By changing the title of a blog post from 2023 to 2024, we achieved:
- A 120% increase in clicks per day to that page.
- The click-through rate increased from 4.16% to 9.94%.
Content refreshes
Content refreshes are great because you can use them to optimize existing content and increase your page’s traffic. Prioritize content refreshes if you’ve completed a content audit and found potential pages for one.
Take a look below at an increase in impressions for one of our blog posts after completing a content refresh:
These were not just empty impressions, either. We saw that this post’s clicks per day increased by 469%. It is still one of our top-performing blog posts!
Embedding YouTube videos to blog content
Videos impact search rankings, but it can be challenging to prove it It’s also no walk in the park trying to convince managers to supplement written content with video content, or to invest in video content on its own.
Well, there’s a straightforward answer: Conduct SEO tests.
One of our most successful tests to date includes video content we produced to complement our written content. Take a look at the example below:
In this test, we selected our blog post about pages being blocked by 4xx issues in Google Search Console, created a video, uploaded it to YouTube, and placed the video at the top of the blog post.
The results speak for themselves:
- A 127% increase in clicks per day to that page.
- The page’s click-through rate went from 4.5% to 8.54%.
- The queries ranking for the page per day went from 17 to 24.
That’s a pretty successful test!
Improving page speed
SEOs value page speed highly, and for good reason. Recent Google search algorithm leaks confirmed that Google uses user experience signals in ranking to some extent, at least partially. However, we don’t know the exact weight the algorithms place on various user engagement metrics.
Running a page speed test is also ideal, but it can be more complicated than it needs to be because you will have to work with development teams. However, a successful page speed test will help you gain management support and demonstrate the value of improving page speed across your site.
Adding internal links
We have found internal links to be one of the biggest needle movers in our internal SEO testing. Just take a look at these results:
Our SEO test on this page was successful, delivering the following results:
- Clicks per day to the page improved from 27 to 38.
- Impressions increased by over 30%.
- The average position for the page improved from 64.47 to 56.81
- The click-through rate had a slight improvement.
The change we made? We added seven new internal links to the page from seven different URLs relevant to the blog post. This resulted in a 40% increase in traffic to the page. That is a good return!
How to analyze your SEO test results
For any SEO test to succeed, you must know how to analyze your SEO test results correctly and be able to learn from your data.
Start by setting clear benchmarks for comparison. These should include:
- Baseline metrics: Collect data before the test begins to establish performance levels for comparison.
- KPIs: Define the specific metrics you want to measure, such as organic traffic, ranking positions, click-through rates, conversions, or engagement rates.
You must also compare your SEO test’s performance over an appropriate period. Use the same duration for the pre-test and post-test periods and factor in potential seasonal trends or events that could impact your data.
For example, compare the four weeks before and after the test. You can adjust for seasonality or major industry shifts as needed.
What do we mean when we say industry shifts? Google’s core updates are a great example. If you find that a confirmed core update happened during your pre-test or post-test period, disregard the test. You can’t be sure whether the results came from your change or an algorithm shift.
One final thing to note: Focus on correlation, not causation.
SEO is often influenced by many factors, which can include the following:
- Algorithm updates
- Competitor activity
- Seasonal changes in search
- Outside factors like breaking news
Avoid jumping to conclusions. While a spike in organic traffic might coincide with your test, it is important to confirm that your changes, not external factors, drove the improvement.
Wrapping things up
SEO testing is a powerful tool for driving measurable improvements to your website’s performance.
Performing thoughtful, data-driven experiments can refine your SEO strategies and help you achieve significant organic traffic growth.
Each test provides clear directions for future SEO work, whether you’re updating meta titles or optimizing page speed. Approach SEO testing systematically by setting clear goals, analyzing the results diligently, and remaining open to iterative learning.
This commitment to constant improvement ensures long-term success and helps you learn and adapt your SEO strategy over time.
