AI Overviews Research: Comparing pre and post-rollout results on 100K keywords

September 2024 Update:
Our recent findings indicate changes in AI Overview behavior. Read our recent research on AI Overviews and Google Ads in search. To learn more about the sources featured in AIOs, check out our July 2024 research on AI Overviews.

During Google’s annual I/O 2024 event on May 14th, the search giant announced its rollout of AI Overviews in the U.S. market, with plans to expand further into more countries. This came just a year after Google’s I/O conference on May 10, 2023, where they announced the integration of generative AI in search. Initially, Google referred to these automatically generated answers as the Search Generative Experience (SGE), but they changed the terminology to AI Overview (AIO).

Since these changes could significantly impact the SEO industry, we aim to ensure SEO specialists and website owners are prepared to make data-driven decisions. 

That’s why we released our powerful AIO Overviews Tracker. This tool helps monitor your keywords’ AIO presence, analyzes sources featured in AI-generated answers, monitors your competitors in AIOs, and much more. We’ve also added AIO SERP feature detection capabilities into our Keyword and Competitive Research Tools. Sign up for the free trial to access these features and launch AIO tracking for your projects. 

We also conducted our first SGE research in February 2024 before the rollout. It was one of the first studies in the industry to analyze AIO in search. In June 2024, we conducted another round of research to see how AI Overviews and the new search landscape have changed after rollout. We used the SE Ranking platform to analyze the same 100,000 keywords, each with various search intents and volumes across 20 niches in the US (New York) region.

Now, let’s move on to the results.

TL;DR:

We’ve created AI Overviews research infographics to visually showcase the most important and interesting findings. Check it out and share it with your network!

Disclaimer:

In the sections below, you’ll find our new research results compared to the data from our previous study (on January 30, 2024). We describe how we conducted this research in the Methodology section at the end of the article. 

The analyses and conclusions presented are based on our observations and data interpretations. These are our best hypotheses, not necessarily the only valid perspectives. These research results should be viewed as a snapshot of Google’s AI Overview behavior during our data collection.

How frequently do AI Overviews show up in search results?

We examined 100,013 terms and found that only 8.71% of searches (8,718 keywords) result in AI Overviews (AIO). A staggering 90.85% of keywords did not trigger AIOs and a minor 0.42% (428 keywords) resulted in a technical error where Google failed to generate an AI Overview.

During our latest analysis, we didn’t encounter a single instance where Google offered the option to generate an AI Overview with the Generate button. Our previous SGE research, conducted before the AIO rollout, found that the Generate button appeared in 45.1% of searches. 

We also observed a significant decrease in the presence of AI Overviews in searches since January 30, 2024: 

• AIO appearances in search have dropped by approximately 52.76%, down from 18,455 to 8,718 keywords.
• The number of searches without AIOs increased by 636.24%, up from 12,342 to 90,867 keywords. 
• The number of searches ending with a technical error decreased by 98.22%, down from 24,105 to 428 keywords.

What is the typical text length of an AI Overview?

After our previous research, we checked for any changes in AIO text length. Our latest findings indicate that the average AIO text length is approximately 4,342 characters. Part of this text is still hidden under the drop-down for users to expand.

Text lengths typically ranged from 3,508 to 5,267 characters.

Compared to our previous SGE research, the average length of text in AI Overviews has increased by approximately 24.59% (from 3,485 to 4,342 characters).

How many sources do AI Overviews feature?

We examined the number of links (also referred to as sources) in AIOs before and after expanding the answer via the drop-down (pre-click and post-click states).

Although we observed a maximum of 19 pre-click links, the most frequently occurring number of pre-click links was 1 link.

After expanding the AI Overview, the highest number of links recorded was 26 links. The most frequently occurring number of post-click links was 4 links.

Compared to our previous SGE research results, we observed a decrease in the number of links present from AIOs: 

• The maximum number of featured pre-click links decreased by approximately 72.86% (from 70 to 19 links) and the biggest number of post-click links decreased by 81.94% (from 144 to 26 links). 
• The most common number of pre-click links remains the same, with just one link. Post-click links decreased twice as much, from 8 to 4.

We expanded our research this time around, and investigated the average number of links in AI Overviews by niche. Our findings show that the average number of pre-click links across researched niches is approximately 2.2, while the average number of post-click links is around 5.5. 

The Healthcare, Legal, and Finance niches lead in the average number of pre-click links. These information-sensitive industries belong to the YMYL category and require more authoritative sourcing even before users expand the AIO. 

The Travel, Relationships, and Sports and Exercise niches have the biggest average number of post-click links. These topics have a broader exploratory nature, where users are likely looking for diverse opinions, detailed explanations, or additional resources.

Do AI Overviews link to domains from the TOP 10 organic search results?

To understand how the relationship between AIOs and traditional organic search results has changed since our previous research, we examined whether AI-generated snippets still include links to domains from the top 10 organic listings and how many.

Out of the 8,718 cases where SERPs included an AI Overview, there were 7,386 instances (84.72%) where AIOs linked to at least one domain from the top 10 organic search results. In 1,332 cases (15.28%), AIOs did not overlap with any of the top organic domains.

In previous SGE research, out of 18,455 SERPs with AIOs, there were 15,786 cases (85.5%) where AIOs linked to at least one domain from the top 10 organic results. In 2,669 cases, (14.5%) showed no overlap. When comparing our current and previous findings, we observed a slight decrease of 0.78%. However, note that in this study, we observed AI Overviews appearing in search results almost half as often as before.

When examining the intersection of links in AI Overviews and traditional top search results (7,386 cases) across different niches, we noticed that the News and Politics niche maintained the highest intersection at 42.96%. The Fashion and Beauty niche exhibited the lowest overlap at 9.02%.

Compared to previous research findings (15,786 cases where links in AIOs overlapped regular search results), the News and Politics niche saw a decrease of 5.07% in link intersection (from 48.03% to 42.96%), while the Fashion and Beauty niche dropped by 1.33% (from 10.35% to 9.02%). The Cars niche saw the biggest drop at 22.53%, while the Education niche saw the biggest increase, 7.97%.

Note that the total number of links and intersected links for each niche changed from our previous research. One reason for this is the decreased number of AIOs in search results.

Which sites show up in AI Overviews?

We analyzed the most frequently linked domains in our current research to understand which types of websites are referred to most in AIOs.

The top 5 linked websites identified are:

• Runnersworld.com (linked 1,200 times)
• Healthline.com (linked 1,114 times)
• Linkedin.com (linked 1,027 times)
• Runrepeat.com (linked 933 times)
• En.wikipedia.org (linked 775 times)

Since our previous SGE research, Linkedin.com and En.wikipedia.org stayed among the most linked domains, but their positions were swapped. Quora.com, Investopedia.com, and Local.google.com dropped out of the top. New players like Runnersworld.com and Healthline.com emerged as top-linked sites. 

In general, the top has been updated by 50% (half of the domains are new to this list). We no longer see Reddit.com and Nerdwallet.com in the top 10 linked domains, Forbes.com stayed in the same position, while Healthline.com and Youtube.com climbed to positions 2 and 6, respectively.

The websites referenced within AIOs still have a substantial online presence and authority. We used the SE Ranking platform to check them. We discovered that their organic keyword count ranges from 119M to 1.2B, while the number of referring domains varies from 5.1M to 9.7M. 

Compared to previous data where the associated organic keyword count varied from about 115M to 950M and the referring domain count spanned from approximately 2.5M to 52M, we observed that the lower line values for organic keywords and domains rose to 3.78% and 104.32%, respectively. This does not necessarily represent eligibility for being featured in AIO but rather reflects shifts across the dataset.

Which niches are more likely to trigger AI Overviews?

To understand the prevalence of AIOs across different industries, we re-investigated 20 niches, each represented by approximately 5,000 keywords.

Our current findings reveal that the Relationships niche is the leader, with 1,331 keywords (26.62%) yielding AIOs. This niche is followed by: 

• Food and Beverage (24.78%)
• Technology (18.11%)
• Business (16.88%) 
• Pets (15.58%)

Our study also highlighted certain niches that triggered significantly fewer AIOs (less than 1%): 

• Travel (0.86%)
• Healthcare (0.44%)
• Legal (0.34%)
• News and Politics (0.24%)
• Career and Jobs (0.14%)
• Cars (0.12%)
• Real Estate (0.12%)

This round, when compared to our previous SGE research, we observed shifts in the niches most likely to generate AIOs. Our earlier analysis showed the Food and Beverage category leading with 33.2% of keywords triggering AI snippets, followed by Business (27.2%), Relationships (25.9%), Ecommerce and Retail (25.7%), and Fashion and Beauty (25.2%).

While the Relationships and Business niches have maintained their positions among the top niches for AIOs, the positions of other categories have changed. Our current findings show, for instance, that Food and Beverage, which previously took the lead, has now dropped to second place, while Technology has emerged as a prominent niche for AIOs.

The Ecommerce and Retail niche, which previously held fourth place, has now dropped to eighth. Niches like Travel, Cars, and Healthcare, which used to generate between 19.76% to 23.5% of SGE snippets, have now dropped to below the 1% mark.

We also identified websites with the most links in SGE snippets by niche:

For a complete list of top-linked websites in each niche, check out this file.

Does the word count in a search query influence the chances of triggering AI Overviews?

Our current findings reveal a clear trend: the more words in a search query, the greater the probability that such a query will receive an AIO. 

Here is a breakdown of our observations:

• Single-word queries triggered AIOs in only 0.8% of cases (3 out of 374 queries).
• Two-word queries had a rate of 5.22% (796 out of 15,256 queries).
• Three-word queries triggered AIOs in 7.17% of cases (2,312 out of 32,238 queries).
• Four-word queries continued their upward trend, with an 8.95% rate (2,394 out of 26,762 queries).
• Five-word queries had a rate of nearly 11.13% (1,614 out of 14,507 queries).
• Six-word queries resulted in AIOs in 13.48% of cases (895 out of 6,493 queries).
• Seven-word queries had a rate of 15.33% (411 out of 2,681 queries).
• Eight-word queries triggered AIOs in 18.28% of cases (194 out of 1,062 queries).
• Nine-word queries resulted in AIOs in 18.40% of cases (85 out of 462 queries).
• Ten-word queries continued their upward trend with a 19.10% rate (34 out of 178 queries).

While our earlier findings indicated a consistent increase in AIO occurrence, with eight and nine-word queries being the leaders, the current data reveals a more pronounced disparity between shorter and longer queries.

This time resulted in a smaller number of instances where AI snippets appeared in search results. The percentages for queries with different word counts are lower compared to previous results.

Does keyword search volume affect AI Overview appearance in SERPs?

Our findings reveal that AIOs appear more frequently for keywords with a search volume of 50 or less.

To aid our research, we segmented keywords according to their monthly search volume values and checked how frequently each group triggers AIO. Here’s the detailed breakdown based on 8,718 searches that ended with AIOs:

• Search volume from 0 to 50: such keywords triggered AIOs 33.21% of the time (2,895 keywords).
• Search volume from 50 to 100: such keywords triggered AIOs 9.46% of the time (825 keywords).
• Search volume from 100 to 200: such keywords triggered AIOs 17.09% of the time (1,490 keywords).
• Search volume from 200 to 300: such keywords triggered AIOs 9.04% of the time (788 keywords).
• Search volume from 300 to 400: such keywords triggered AIOs 8.81% of the time (768 keywords).
• Search volume from 400 to 500: such keywords triggered AIOs 4.39% of the time (383 keywords).
• Search volume from 500 to 1,000: such keywords triggered AIOs 8.29% of the time (723 keywords).
• Search volume from 1,000 to 10,000: such keywords triggered AIOs 8.09% of the time (705 keywords).
• Search volume from 10,000 to 100,000: such keywords triggered AIOs 1.39% of the time (121 keywords).
• Search volume of 100,000 or above: such keywords triggered AIOs 0.23% of the time (20 keywords).

Do AI Overviews appear alongside featured snippets?

To understand the evolution in interplay between traditional SERP features and AI Overviews, we investigated how often featured snippets appeared alongside AIOs in search results.

Our current findings indicate that when AIOs are present, featured snippets appear 45.39% of the time (3,957 out of 8,718 cases).

This represents a significant increase compared to our previous SGE research, where featured snippets co-occurred with AI Overviews only 23.03% of the time (4,251 out of 18,455 cases). Note that during our current research, there were fewer overall cases where an AI Overview appeared (8,718) compared to the previous analysis (18,455). But even with fewer AI snippets, the percentage increased.

We also noticed that when AIOs were not present, featured snippets appeared less frequently; only 8.61% of the time (7,856 out of 91,295 keywords). This is significantly lower compared to searches where both AIOs and featured snippets appeared together, which occurred 45.39% of the time. This difference may indicate that AI Overviews and featured snippets might be triggered by similar types of keywords. 

We also analyzed whether the link from the featured snippet overlapped with the source in AIO. 

Out of the 3,957 instances in which featured snippets and AIOs appeared together, 2,445 cases (61.79%) saw a link match between them.

We also checked which domains get links from both featured snippets and AI Overviews simultaneously. Here are the top 20 domains:

To see the 10 websites being included in featured snippets and AIO simultaneously, and by niche, check out this file.

Does keyword CPC affect AI Overview appearance in SERPs?

We wanted to understand how cost-per-click affects the presence of AIOs. Our findings revealed that AIOs appear more frequently for keywords with a CPC of $0.5 or less.

We categorized keywords based on their CPC. Here is what we found:

• CPC from $0 to $0.5: These queries ended with AIOs 53.65% of the time (4,677 keywords).
• CPC from $0.5 to $1: These queries ended with AIOs 12.31% of the time (1,073 keywords).
• CPC from $1 to $1.5: These queries ended with AIOs 13.13% of the time (1,145 keywords).
• CPC from $1.5 to $2: These queries ended with AIOs 8.91% of the time (777 keywords).
• CPC from $2 to $2.5: These queries ended with AIOs 4.53% of the time (395 keywords).
• CPC from $2.5 to $3: These queries ended with AIOs 3.01% of the time (262 keywords).
• CPC from $3 to $4: These queries ended with AIOs 2.44% of the time (213 keywords).
• CPC from $4 to $5: These queries ended with AIOs 1.02% of the time (89 keywords).
• CPC from $5 to $6: These queries ended with AIOs 0.42% of the time (37 keywords).
• CPC at more than $6: These queries ended with AIOs 0.57% of the time (50 keywords).

Do ads show up with AI Overviews, and where?

Our latest data indicates that when AIOs are present (8,718 cases):

• Ads appear at the top of the SERP in 48.58% of cases (4,235 times).
• Ads appear at the bottom of the SERP in 28.71% of cases (2,503 times).
• Shopping ads (only) appear in 9.51% of cases (829 times).
• No ads accompany AI Overviews in 13.20% of cases (1,151 times).

In our previous SGE research, of the 18,455 cases when AIOs were present in the SERP ads appeared at the top of search results 23.19% of the time and at the bottom 35.22% of the time. Our current data shows a shift in ad placement, with ads at the top occurring more frequently.

The presence of shopping ads alongside AIOs has decreased, occurring 9.51% of the time compared to the 14.41% occurrence in our previous analysis.

As for shopping ad placement in the SERP, the trend spotted during our previous research continues: shopping ads appear above the AIO snippet more often than in other locations. In this study, we found shopping adds above AIOs 1,800 times (81.52% of cases). The previous research showed shopping ads above AIOs 2,969 times (80.72%). However, the previous study had more AIOs overall (18,455). 

Placements in the sidebar occurred more often this time than last—241 times (10.91%) compared to 207 times (5.63%). This time, placements in the sidebar became even more widespread than ads below the AIO, which appeared in only 166 cases (7.52%), making it the least common placement.

We observed the following when analyzing the combinations of ad types and their placement relative to AIOs:

• In 2,856 cases, only ads at the top of the SERP appeared alongside AIOs.
• In 1,379 cases, both shopping ads and ads at the top of the SERP accompanied AIOs.
• In 829 cases, only shopping ads appeared alongside AIOs.

We also investigated niches that tend to have specific ad types when AIOs are present:

• Leaders in shopping ads only: Ecommerce and Retail (233 cases), Food and Beverage (178 cases), Fashion and Beauty (168 cases), and Pets (156 cases).
• Leaders with a combination of shopping ads and ads at the top: Fashion and Beauty (455 cases), Pets (382 cases), Ecommerce and Retail (279 cases), and Fashion and Beauty (137 cases).
• Leaders with ads at the top only: Relationships (594 cases), Business (481 cases), Technology (382 cases), and Self-Care and Wellness (305 cases).

Keep in mind, though, that some niches had fewer AIOs appearing in search results. Therefore, fewer ads were detected and presented on the graph.

Compared to our previous research findings, we noticed changes in ad types appearing with AIO across niches. For instance, the Fashion and Beauty category, which previously had more shopping ads only, now more likely shows a mix of shopping and top ads. The same trend applies to the Ecommerce and Retail and Pets niches.

Methodology

The goal of this research study was to understand the behavior of AI Overviews in Google search results, thereby helping SEO specialists and website owners prepare for the new SERP. We want you to stay updated so you can make better business decisions. 

We analyzed 100,013 keywords across 20 niches, each covering different user interests:

1. Business
2. Career and Jobs
3. Cars
4. Ecommerce and Retail
5. Education
6. Entertainment and Hobbies
7. Fashion and Beauty
8. Finance
9. Food and Beverage
10. Healthcare
11. Insurance
12. Legal
13. News and Politics
14. Pets
15. Real Estate
16. Relationships
17. Self-Care and Wellness
18. Sports and Exercise
19. Technology
20. Travel

Each niche included around 5,000 keywords, varying in search intent, volume, and word count (short, medium, and long-tail keywords). 

We used the SE Ranking platform to conduct our study, creating projects for each niche. We also applied our AI Overview tracking functionality to capture and analyze AI-generated answers in search results. Here is how we set our analysis parameters:

• Search Engine: Google
• Location: USA
• Region: New York, New York, United States
• Language: English (en)

We collected this data using the Google Chrome browser on a PC running Ubuntu, with all sessions logged into Google accounts based in New York. We also used accounts with the Google Labs experiment enabled. While this affects our results, it provides insight into the direction Google search is heading and how it might appear in actual searches soon. In our next study, we’ll aim to use accounts without Labs data to provide a more current view of search results.

To ensure data validity and impartiality, we disabled any personalization features in the accounts used to collect data.

We executed the data collection process as a one-time check on June 3, 2024.

Wrapping Up

Although we hope you found our analysis insightful, the story doesn’t end here. This is only the beginning. With Google continuously enhancing AI Overview features, like the recently spotted travel planning capabilities and the option to simplify AI-generated content, the situation is becoming even more dynamic. We all need to stay vigilant and monitor these updates.

In our next research phase, we’ll focus on the sources (links) featured within AI Overviews, a key focus of our new AI Overviews Tracker. Make sure to sign up for our 14-day free trial to get access to it and start tracking and monitoring AIOs for your projects. Google does not provide any official reporting or analytics for its generative AI search features, so you will need a reliable instrument with up-to-date and accurate data to track your performance and optimize for the new search.

Stay tuned for our upcoming research. Follow our news to stay informed about the latest trends and updates.

Written by

Yevheniia Khromova

Yevheniia is a content marketer and copywriter at SE Ranking. She is a fan of SEO, digital marketing, and content creation. Yevheniia believes simplicity is key as it helps reduce the clutter in our heads and focus on what matters. This is the principle she uses in her writing. She also enjoys cooking, learning new spices and flavors, traveling, and playing board games.