Google Reveals Inner Workings of Query Fan-Out in AI Mode

Google’s VP of Product for Search, Robby Stein, recently provided new insights into the mechanics of query fan-out in AI Mode during an interview. While the existence of this technique has been previously mentioned in Google’s blog posts, Stein's comments offer a deeper dive into how it operates and its practical implications.

How Query Fan-Out Works

When you enter a question in Google’s AI Mode, the system leverages a large language model (LLM) to interpret your query. This interpretation isn’t just a straightforward search; instead, the LLM "fans out" multiple related searches to Google’s infrastructure. These searches can include topics that you didn't explicitly mention.

For example, if you ask, “Things to do in Nashville with a group,” the system might generate additional queries like:

• Great restaurants in Nashville
• Best bars in Nashville
• Activities for kids in Nashville

Stein explained:

"It may think of a bunch of questions like great restaurants, great bars, things to do if you have kids, and it’ll start Googling basically."

The system then combines the results from these multiple queries into a single, cohesive response that includes links. This functionality is active in AI Mode, Deep Search, and some AI Overview experiences.

Scale and Scope

AI-powered search features, including query fan-out, now serve approximately 1.5 billion users each month. These features support both text-based and multimodal input, making them versatile for a wide range of queries.

The underlying data sources are extensive:

• Traditional web results
• Real-time systems like Google’s Shopping Graph (which updates 2 billion times per hour)

Stein referred to Google Search as “the largest AI product in the world,” highlighting its massive scale and impact.

Deep Search Behavior

For more complex queries that require deeper reasoning, a feature called Deep Search may be triggered. This process can involve dozens or even hundreds of background queries and might take several minutes to complete.

Stein shared an example of using Deep Search to research home safes, which involved unfamiliar factors like fire resistance ratings and insurance implications:

"It spent, I don’t know, like a few minutes looking up information and it gave me this incredible response. Here are how the ratings would work and here are specific safes you can consider and here’s links and reviews to click on to dig deeper."

AI Mode’s Use of Internal Tools

AI Mode also has access to internal Google tools, such as Google Finance and other structured data systems. For instance, if you’re comparing stocks, the system might use:

• Real-time stock prices from Google Finance
• Historical performance data
• Analyst ratings and news articles

This integration allows for more comprehensive and accurate responses, leveraging Google’s extensive internal resources.

Why It Matters to Practitioners

For developers and engineers working on search and AI systems, understanding query fan-out is crucial. Here are a few key takeaways:

• Enhanced Query Interpretation: The ability to generate multiple related queries can lead to more comprehensive and relevant results.
• Scalability and Performance: Handling billions of users and real-time updates requires robust infrastructure and efficient data processing.
• Integration with Internal Tools: Access to internal systems like Google Finance can enrich the quality of responses, making AI Mode a powerful tool for complex queries.

By leveraging these techniques, Google is pushing the boundaries of what’s possible in search and AI, offering users more sophisticated and useful experiences.