Which Search Index Does Each AI Engine Use? A 2026 Reference

As of October 29, 2026, only three vendors clearly document where their AI answers get web results: AI Overviews, AI Mode and the Gemini app ground in Google’s search index, Copilot grounds in Bing, and Perplexity runs its own index covering hundreds of billions of webpages. ChatGPT uses several suppliers: OpenAI names Bing as a search provider, Microsoft says its Web IQ service serves ChatGPT, Yelp licenses local data to OpenAI, and a vendor investigation found evidence of an OpenAI-built index. Anthropic has not named Claude’s search provider, but Brave Search was on its subprocessor list as of July 2026, and one study found 79.2% of Claude’s cited URLs in Brave’s top 10. xAI and Meta disclose the least: Grok searches the web and X without a named provider, and the latest provider statement we found from Meta, naming Google and Bing, dates to 2024.

Most AI visibility advice treats “the AI engines” as one system. They are not: each sits on a different index with a different crawler, and a rule that helps one can quietly cut you out of another. This reference separates what vendors say from what others infer.

How to read this reference

We grade every claim on three levels. This is our own framework, not an industry standard:

• Documented: the vendor states it in its docs or on its official blog.
• Reported: a news outlet, partner or listing names it, but the vendor gives no detail.
• Inferred: a third party deduces it from tests, traffic data or result overlap.

An inferred link can change overnight without an announcement. Documented does not mean exclusive either: an engine can name one provider and still use others. And most vendors document their crawlers far better than their indexes, which works in your favor, because crawler access is the part you control.

ChatGPT: several suppliers, plus evidence of its own index

The documented core is Bing. OpenAI’s help center says ChatGPT search may share queries “with third-party search providers such as Bing.” Microsoft wrote in June 2026 that “Microsoft Web IQ is already used by many leading AI assistants like Copilot and ChatGPT,” and InfoWorld describes Web IQ as building “on the massive Bing search index.”

Beyond Bing, the picture is reported or inferred. Axios reported on July 23, 2026 that Yelp is licensing reviews, photos and other business information to OpenAI for ChatGPT answers. For products, OpenAI’s feed specification invites merchants to “Submit product data for discovery in ChatGPT.” On September 4, 2026, Peec AI, a monitoring vendor, reported a result_source field in ChatGPT’s server events between May 21 and July 21, 2026, carrying four values: Labrador, Bright, Oxylabs and SERP. Peec reads Labrador as OpenAI’s own family of indexes, from general web to verticals such as news and shopping, and says ChatGPT combines “at least eight other providers,” including Google results gathered through scraping services. OpenAI has not confirmed any of this; the full evidence is in ChatGPT’s own search index.

OpenAI’s crawler docs are clear. OAI-SearchBot is the search crawler, GPTBot is for training, and ChatGPT-User handles fetches a person triggers. Sites that opt out of OAI-SearchBot “will not be shown in ChatGPT search answers, though can still appear as navigational links.” A robots.txt change takes about 24 hours to reach search results, and ChatGPT-User “is not used to determine whether content may appear in Search.”

Google and Microsoft: answers built on their own indexes

Google is the cleanest case. Its AI features documentation says a page must be “indexed and eligible to be shown in Google Search with a snippet” to appear as a supporting link, and “There are no additional requirements to appear in AI Overviews or AI Mode.” Googlebot’s robots.txt rules govern crawling for Search, AI features included. The Gemini app draws on the same index: Google’s crawler documentation, in wording Search Engine Journal reported in April 2025, describes grounding in Gemini Apps as providing content from the Google Search index to the model at prompt time.

The same page holds the one trap. Google-Extended is a robots.txt token without its own user agent string, not a separate crawler, and it controls whether Google may use your content to train Gemini models and to ground answers in Gemini Apps and on Vertex AI. Google says it “does not impact a site’s inclusion in Google Search nor is it used as a ranking signal.” So blocking Google-Extended keeps you eligible for AI Overviews and AI Mode, but by Google’s own description it also covers grounding in the Gemini app.

Microsoft’s documentation for Copilot in organizations says Copilot “generates a search query that it sends to the Bing search service” to ground its answers, and Microsoft names Copilot among Web IQ’s users. The AI Performance report in Bing Webmaster Tools, launched in public preview on February 10, 2026, shows how content appears “across Microsoft Copilot, AI-generated summaries in Bing, and select partner integrations.” The crawler is bingbot. Microsoft also says IndexNow helps AI systems reference the most current version of a page, though IndexNow’s own documentation says it “does not guarantee that web pages will be crawled or indexed.”

Perplexity, Claude, Grok and Meta AI

Perplexity describes “an index covering hundreds of billions of webpages” and says its Search API, launched in September 2025, runs on “the same global-scale infrastructure that powers Perplexity’s public answer engine.” It documents two agents. PerplexityBot is “designed to surface and link websites in search results on Perplexity” and is not used for model training. Perplexity-User fetches pages for user questions and “generally ignores robots.txt rules.” Changes can take up to 24 hours. We found no current Perplexity statement naming an outside search provider.

Claude rests mostly on reports and inference. Anthropic’s web search docs say only that “The API runs the searches.” In March 2025, TechCrunch reported that Anthropic had added Brave Search to its subprocessor list, and developer Simon Willison found a “BraveSearchParams” parameter in Claude’s search function. Xponent21 reported in July 2026 that Brave was still listed, alongside turbopuffer, a search database; we could not open the list ourselves. The strongest data point comes from Profound: across roughly 35,000 URLs tied to 400 queries, 79.2% of Claude’s cited URLs also appeared in Brave’s top 10, which the authors call “a strong observed match, not proof.” Anthropic also runs its own Claude-SearchBot, and its help page has said since at least February 2026 that blocking it “may reduce your site’s visibility and accuracy in user search results.” Claude-User handles user requests, ClaudeBot handles training, and Claude skips search on many prompts, as covered in our look at how often Claude answers from memory.

Grok. xAI’s docs, updated October 10, 2026, describe a Web Search tool that lets Grok “search the web in real-time and browse web pages,” plus an X Search tool for “X posts, users, and threads.” xAI names no web index or provider, and we found no xAI documentation of a crawler user-agent, IP list or robots.txt token. Bot directories list names such as GrokBot, but StackFox reported in February 2026 that it had never seen them in server logs. Treat any Grok crawler rule as unverified.

Meta AI and Muse. The latest provider statement from Meta that we found dates to April 2024: Search Engine Journal reported Meta’s announcement that Meta AI would show real-time results from Google and Bing. In October 2024, Social Media Today, citing The Information, reported that Meta was building its own search engine and crawler. Meta now documents Meta-WebIndexer; according to PPC Land, Meta’s crawler page says allowing it “helps Meta cite and link to that content in Meta AI’s responses.” Meta-ExternalAgent is a separate, training-related crawler. For Muse, launched September 8, 2026, Profound’s exploratory 50-prompt test found it browses merchant sites and “separately queries a Meta product catalog in parallel,” naming no search engine.

What each engine needs to find you

Work through these in order:

1. Audit robots.txt token by token. OpenAI and Anthropic document separate tokens for search, training and user fetches, and Perplexity separates search from user fetches. Blocking GPTBot or ClaudeBot does not block OAI-SearchBot or Claude-SearchBot, and the reverse is also true. The full policy logic is in should you allow or block AI crawlers.

2. Protect Googlebot and bingbot first. They feed the most engines. Brave says “if a domain or page is not crawlable by Googlebot, then Brave Search’s bot will not crawl it either,” and it sends no distinct user agent, so you cannot allow Brave separately. A Googlebot block can therefore reach Claude through Brave, which is our inference from the evidence above.

3. Decide Google-Extended with grounding in mind. It is not only a training switch. By Google’s description, it also covers grounding in the Gemini app.

4. Submit where submission exists. That means Google Search Console, Bing Webmaster Tools with IndexNow, Brave’s form for re-fetching updated pages, and OpenAI’s product feed for merchants. We found no web page submission for ChatGPT, Claude, Perplexity, Grok or Meta AI.

5. Verify in logs against published IP lists. OpenAI publishes openai.com/searchbot.json, Perplexity publishes www.perplexity.com/perplexitybot.json, and Anthropic publishes claude.com/crawling/bots.json. You want 200 responses on pages you want cited; a bot name without a matching IP is unverified.

6. Cover the licensed channels. Keep Yelp listings accurate for ChatGPT local answers, keep product data clean for feeds and catalogs, and treat X as a source surface for Grok.

7. Re-check every quarter. Inferred links move first, and vendor crawler pages change without notice.

How Rankry fits

Knowing the index tells you where you are eligible, not whether you are cited. Rankry tracks ChatGPT, Claude, Gemini, Perplexity, Grok and Google AI Mode / AI Overviews on a stable set of buyer-intent prompts every day, recording mentions, rank and cited sources, so you can see whether an engine-specific fix worked. Its AI traffic tracking shows the visits AI platforms send to your site; the crawler check in step 5 stays in your server logs. Copilot and Meta Muse can be added to the tracked engines, and for Copilot, Bing Webmaster Tools’ AI Performance report adds the first-party view. For how source selection works once you are indexed, see where AI actually gets its answers.

FAQ

Does ChatGPT use Bing or Google for search? Bing is documented; Google rests on third-party evidence. OpenAI names Bing as a third-party search provider, and Microsoft says its Web IQ service, which InfoWorld describes as built on Bing, serves ChatGPT. Peec AI’s September 2026 investigation adds Google results gathered through scraping services, licensed partners such as Yelp, and an OpenAI-built index it calls Labrador. OpenAI has not confirmed the Peec findings, so treat the Google link and the own index as inferred.

Does Claude use Brave Search? Anthropic’s docs do not name a search provider. The evidence is indirect but consistent: Brave Search was added to Anthropic’s subprocessor list in March 2025, a “BraveSearchParams” parameter appeared in Claude’s search function, and Profound found 79.2% of Claude’s cited URLs in Brave’s top 10 across about 35,000 URLs. Anthropic also runs Claude-SearchBot, so Brave is likely not the only input.

Does blocking Google-Extended remove me from AI Overviews? No. Google says Google-Extended does not affect inclusion in Google Search or ranking, and AI Overviews and AI Mode only require that a page be indexed and eligible for a snippet. But Google-Extended does cover Gemini training and grounding in the Gemini app, so blocking it has a cost there.

Can I submit my site to AI search engines directly? Only to some. Google Search Console covers the index behind AI Overviews, AI Mode and Gemini. Bing Webmaster Tools and IndexNow cover Copilot and likely, through Web IQ, part of ChatGPT. Brave offers a form to re-fetch updated pages. OpenAI accepts product feeds, not web pages. We found no web page submission for Claude, Perplexity, Grok or Meta AI.

Does Grok have a crawler I can allow or block? Not one xAI documents. xAI’s docs describe Web Search and X Search tools but name no crawler user-agent, IP list or robots.txt token. Third-party directories list names such as GrokBot, but they are unverified. If Grok visibility matters, focus on being findable in general web indexes and active on X, and do not rely on Grok-specific robots.txt rules.

Sources

1. ChatGPT search, OpenAI Help Center, n.d.
2. Building a new AI economy that creates value for everyone, Microsoft Advertising Blog, June 17, 2026
3. Microsoft Web IQ: Ground your AI agents with up-to-date web data, InfoWorld, August 6, 2026
4. Exclusive: Yelp deal pushes local reviews into ChatGPT, Axios via Yahoo Finance, July 23, 2026
5. Products, Agentic Commerce, OpenAI Developers, n.d.
6. ChatGPT built its own search index, Peec AI, September 4, 2026
7. Overview of OpenAI Crawlers, OpenAI, n.d.
8. AI features and your website, Google Search Central, updated December 10, 2025
9. Google’s common crawlers (Google-Extended), Google Search Central, n.d.; quoted wording confirmed in sources 10 and 11
10. Google Updates Gemini/Vertex AI User Agent Documentation, Search Engine Journal, April 28, 2025
11. Google-Extended Bot (Token) Does Not Affect Google Search, Search Engine Roundtable, February 9, 2024
12. Data, privacy, and security for web search in Microsoft Copilot and Microsoft Copilot Chat, Microsoft Learn, updated August 18, 2026
13. Introducing AI Performance in Bing Webmaster Tools Public Preview, Microsoft Bing Webmaster Blog, February 10, 2026
14. How to add IndexNow to your website, Bing Webmaster Tools, n.d.
15. Bingbot is coming to town, Microsoft Bing Webmaster Blog, September 2010
16. Introducing the Perplexity Search API, Perplexity, September 2025
17. Perplexity Says New Search API Accesses Same Infrastructure as Public Search Engine, PYMNTS, September 26, 2025
18. Perplexity Crawlers, Perplexity Docs, n.d.
19. Web search tool, Claude Platform Docs, n.d.
20. Anthropic appears to be using Brave to power web search for its Claude chatbot, TechCrunch, March 21, 2025
21. Anthropic Lists Two Web-Search Subprocessors Behind Claude, Xponent21, July 2, 2026
22. The state of AEO in 2026: Claude is not ChatGPT, Josh Blyskal and Jasman Singh (Profound data), July 22, 2026
23. Does Anthropic crawl data from the web, and how can site owners block the crawler?, Claude Help Center, n.d.; bot descriptions confirmed in source 24
24. Anthropic’s Claude Bots Make Robots.txt Decisions More Granular, Search Engine Journal, February 25, 2026
25. ClaudeBot: What It Crawls and How to Allow or Block It, Chudi.dev, October 17, 2026
26. Brave Search Crawler, Brave Search Help, n.d.
27. Web Search, xAI Docs, updated October 10, 2026
28. Tools Overview, xAI Docs, updated October 10, 2026
29. Grok Bot User Agent: Why You Can’t Block xAI’s Crawler, StackFox, February 2026
30. Meta Integrates Google & Bing Search Results Into AI Assistant, Search Engine Journal, April 18, 2024
31. Meta Is Developing a Search Engine to Power Its AI Chatbot, Social Media Today, October 28, 2024
32. Explaining Meta-WebIndexer, PPC Land, August 29, 2026
33. Introducing Muse: The World’s First Personal AI Agent Built for Everyone, Meta, September 8, 2026
34. The era of personal agents: Muse and Instinct, Allison Huang, Profound, October 2, 2026

---

See which engines actually cite you, and which ones send you visitors, instead of guessing from one index. Start a free 7-day Rankry trial, no card required.