Google AI Overview in the Heavy Machinery Industry - analysis
During the Google I/O presentation on May 14, Google introduced a new feature called AI Overview (AIO) in its search engine, which was previously known as the Search Generative Experience (SGE) during an earlier testing stage.
We analyzed queries within AIO results to examine how Google’s responses are presented in the SERP and which links are displayed as sources for AI-generated answers.
We are publishing our study to help explain how Google presents links to websites in this new feature, which will appear as the top organic position for a large percentage of queries.
What is the Google AI Overview feature?
The AI Overview is a response to a user query generated by an AI model named Gemini. In addition to the answer, Google also displays sources with links to websites. In the set of phrases we analyzed, Google showed between one and twelve links. In other studies, mentioned later in the text, it showed up to 20 links.
The AIO response can appear in various formats and be presented for different query intents. It shows up for informational, transactional, and local queries. Depending on the topic, AIO appears in search results for a few to several dozen percent of queries.
AI Overview is essentially an improved version of the featured snippet module, providing a ready-made answer to a user’s query selected by Google from one of the search results. Like the featured snippet, AIO appears only for certain queries and always as the first organic result in the search engine. Currently, it is available only to users in the United States but is expected to be introduced to other countries, although no timeline has been provided.
Why was AI Overview introduced? Google’s perspective
AI Overview can respond to longer and more complex queries, which is intended to increase the frequency of using Google Search and user satisfaction. Google states that this feature leads users to visit a greater variety of websites, and the links included in AI Overview receive more clicks compared to traditional search results. Google aims to attract more users and engage them more deeply by enabling a new, more advanced way of using the search engine. Unfortunately, we cannot currently track interactions with AIO links in Google Search Console, so we cannot verify the official claims of the Google team.
Google emphasizes:
“And we’re encouraged to see not only an increase in Search usage, but also an increase in user satisfaction.” Link
Study results
- AI Overviews were generated for 26% of queries in the selected niche (heavy machinery) in June and 30% in July.
- On average, Google showed 5.3 links (sources) per query.
- Certain words trigger the AIO feature more often. Among them are “specification,” “what,” and “horsepower.”
- The percentage of links in AI Overviews that also appeared in the TOP10 search results was 68%, while for the TOP100 this was 89%.
- PDF files appeared as sources in 16.83% of AIO responses.
- YouTube videos appeared in 21.29% of AIO modules.
- After a month, 84% of phrases from June still generated AIO responses.
How we collected and analyzed the data - methodology
We analyzed the TOP100 organic search results, focusing on one niche, the heavy machinery industry. Other studies, mentioned later, considered the TOP10 or TOP50 results.
Data was collected in the first half of June from Google accounts without experimental features from Google Search Labs enabled. Therefore, there is no risk that the data might be presented differently than to regular users.
When checking the TOP100 SERPs, we did not include links appearing in People Also Ask modules, Video and Images boxes, or ads.
Due to the niche we chose and specialized queries, Google occasionally did not return 100 results for a query but, for example, only 60.
We selected both less and more popular phrases from Ahrefs and Google Search Console tools and also created our own longer queries.
What did an average response look like?
AIO returned both very short, few-sentence responses and extensive texts with several paragraphs. Short answers might include just one source link, while the most comprehensive ones could have up to twelve links. On average, Google showed 5.3 sources per query.
Below you can see a very short response with one link shown as the source for both the first and second parts of the answer.
Only the links for the first part of the answer in AIO were displayed immediately; to view the rest, an additional button needed to be clicked, as shown below.
How often do AI Overviews appear?
The percentage of responses for which Google generates AIO is one of the most frequently reported metrics. However, it is important to note that this number greatly depends on the selected niche and the specific phrases used.
For the keywords we studied, AI Overview responses appeared for 26% of queries.
In addition to data from GSC and Ahrefs, we included longer, more complex queries that users might enter to interact with AIO. Such long-tail phrases are likely to appear more frequently, as the search engine better understands the context and can create more personalized responses.
Studies by Advanced Web Ranking (AWR), SE Ranking, and Onely show that AI-generated responses are highly dependent on the chosen niche.
An additional factor that likely contributed to the high percentage of AIO responses is the lower search volume characteristic of queries in specialized niches. SE Ranking found that for queries with a search volume of up to 50 searches per month, AIO appears in 33% of cases for their set of keywords.
Another conclusion is that longer queries more often generate AIO responses. In our study, the average number of words in the examined phrase was 4.55. For comparison, the average number of words in the TOP100 1,000 phrases from Google Search Console in this industry is 3.4.
Which words more frequently trigger AI Overview?
We also checked if certain words trigger the AI Overview module more frequently. Words like ‘how’ generated AIO response in 23% of cases, ‘weigh’ in 29%, ‘specification’ in 31%, ‘what’ in 35%, and the phrase “horsepower” triggered AIO in 78% of queries.
AI Overviews and Featured Snippet
It is worth noting that generating AIO responses does not interfere with the appearance of the Featured Snippet. It appeared simultaneously with AIO in 24.26% of cases. Other modules, such as People Also Ask or images, also appeared in the SERPs. Thus, contrary to some expectations, the AIO module does not replace the Featured Snippet or affect its frequency.
Interestingly, there is no correlation between appearing in the Featured Snippet and being a source in AIO. Websites that frequently appeared in the Featured Snippet might not be sources for AIO.
How frequently do links in AIO appear in the TOP100 search results?
The main thesis we aimed to verify was whether being present in the TOP10 or TOP100 search results for a given query is necessary to appear in AIO. The answer is no (as of the article’s publication date, July 15, 2024). This finding is notable as it confirms that the algorithm used to present links in AIO operates differently from the one typically used by Google to generate search results.
The percentage of links in AIO that simultaneously appeared in the TOP10 search results was 68%, while for the TOP100, this figure was 89%. We verified whether these were exactly the same links. This distinction is important because some similar studies report these percentages but only refer to whether the domain appears in the TOP10.
It is also worth noting that Google sometimes selected one link from a domain for AIO, while a different but similar address from the same domain appeared in the TOP100 for that exact query.
Additionally, AIO might use data from version A of a page, while the top 100 might feature the same subpage in version B, where “version A” and “version B” refer to different language versions of the page. In such cases, we considered these as different links to maintain methodological consistency.
Types of sources in AI Overviews
In addition to website links, the AIO module also frequently used PDF files, which appeared as sources in 16.83% of responses.
We also found it interesting that YouTube videos appeared as sources, in 21.29% of cases, along with TikTok videos and videos embedded on websites.
What changed after a month?
At the beginning of July, we checked how many phrases that generated AIO responses a month earlier still showed them. It turned out that this was the case for 84% of the phrases, meaning that 16% of phrases no longer returned AI-generated answers. However, new phrases that did not return AIO a month ago did appear. As a result, the overall number of AIO responses in our set of phrases increased from 26% to 30%. For some terms like “specification,” the number of phrases decreased, while for others, such as “how,” it increased, and for some, it remained unchanged. It is evident that this is a testing phase for Google and that much will continue to change.
Interesting statistics from other studies
The Onely team shared some interesting conclusions. They tracked changes in the module already during the SGE stage. In May, they presented data showing how Google reduced the frequency of AI-generated responses over the preceding months. At the beginning of May, only about 14% of phrases automatically returned AI Overview results, while an additional 28% could show AI results by pressing the “generate” button. This manual option is no longer available for AI Overview. As shown in the chart below, the percentage increase in non-AI response was very high in the months directly preceding the official introduction of AI Overview.
Broader research has also been published by Advanced Web Ranking (AWR) and SE Ranking.
In the Advanced Web Ranking studies, the AI Overview (AIO) appeared in 12.4% of queries, while in SE Ranking, in 8.71%. This is a significant difference compared to our results (26.10%) and clearly demonstrates how the frequency of AIO generation varies depending on the niche and keywords selection.
The Featured Snippet appeared alongside AIO in 59.5% of cases (AWR) and 45.39% (SE Ranking), whereas in our study, it was 24.26%.
The average number of links provided in the AIO response also varied. We found an average of 5.3. Advanced Web Ranking reported an average of 7.2, while SE Ranking calculated separate averages: 2.2 links for the first part of the AIO and 5.5 links for the subsequent part of the response.
In the AWR study, 33.4% of the links from the AI Overview were present in the TOP10 organic results for those queries. SE Ranking, on the other hand, found a match of 84.72%, although this was based on whether the same domain appeared in the first organic results, not specific links. In our study, 67% of the links from AIO were in the TOP10.
What features is Google planning to add to AI Overview in the near future?
During the I/O presentation, Google announced plans to add new features to AI Overview later this year. These include the ability to generate travel or diet plans. For example, users will be able to request a weekly diet plan that excludes dairy products and follows a Mediterranean style. Additionally, users will be able to add notes to the proposed plans to customize them further. Such diet or travel plans can be conveniently exported to other Google products like Gmail or Docs.
Should I prepare for the introduction of AI Overview even if my site is currently well-positioned?
Having the highest positions in current search results does not guarantee that site will appear as a source in AI overview.
What we know for sure is that the algorithm for AI overview is entirely different from the ranking algorithm used before. Although being in the top ten search results increases the likelihood of appearing as a source for AI overview, often the sources are pages not listed in the TOP10. This means that the algorithm scans and classifies content differently.
Previously, Google selected documents that focused on a given topic entirely. However, for AI Overview, the entire document no longer holds as much significance.
AI Overview can base its responses on content fragments from articles that may only briefly mention the topic rather than discuss it in depth. These fragments can be located anywhere in the article – whether in the middle or at the end – and are still considered relevant.
Therefore, it is important to monitor the performance of the new algorithm and adjust your site’s content accordingly to align with this new functionality.
How can you optimize your site to appear in AI Overview?
The first experiments on content and its editing have helped identify factors that increase the likelihood of appearing as a source in AI responses:
- Content on the site should answer user queries concisely and clearly.
- Google must be able to read, render, and index the content.
- More complex issues should be broken down into smaller, specific steps (e.g., bullet-point lists are useful).
- Longer content should include concise summaries that highlight key information.
- Google is less likely to use pages that present content via JavaScript, and when it does, it prefers faster and lighter pages. This study is based on content appearing in SGE and has not yet been confirmed for AIO.
Can you block the display in the AI Overview?
The way to block your site from appearing in AI Overview is to add the ‘nosnippet’ tag in the page header or block the subpage from indexing in search results (in robots.txt or through the ‘noindex’ tag). However, this approach is not recommended, as it limits your visibility and potential traffic from the search engines.
Summary
Now is the time to adjust the content for the new module in the Google search engine. We know that AI Overview functions differently than the previous algorithm and favors a slightly different way of presenting information. To prepare for these changes, start with an audit of your site’s content. Decide which content should be simplified, which summaries need to be written, and which should be presented in concise lists.
AI Overview will change how search results are displayed and how users interact with the search engine. It’s important to closely monitor upcoming changes and continuously adjust your site to meet the evolving needs of users.
Contact us!
We will analyse the situation of your website, possible scenarios and the impact of AI overviews on your organic traffic and plan actions.
Author: Kornel Osowski with ideahills team