Claude finds sources by running live web searches against an external index, by all public evidence Brave Search’s independent 40-billion-page index, then fetching the most promising pages, filtering their content, and writing an answer with inline citations linking to each page it used. Claude decides on its own when a question needs fresh information. Anthropic has never officially named its search provider, but its own subprocessor list, exposed API parameters, and independent citation-overlap tests all point at Brave, with a second, quieter entry on that list, TurboPuffer, suggesting the pipeline has a retrieval layer nobody fully maps yet.
If you care whether Claude cites your brand, this mechanic is the whole game. Claude’s view of the live web is not Google’s view. A page that ranks first on Google can be invisible to Claude, and a page Google buries can be Claude’s favorite source. Here is how the pipeline actually works, step by step, with the evidence behind each claim.
The pipeline, end to end
When you ask Claude a question, the first decision is whether to search at all. Web search is a server-side tool that the model invokes at its own discretion. There is no separate browse button in the flow: Claude weighs whether the answer needs information past its training data, current prices, recent events, live status, and if so it searches. Questions it can answer from training, it answers without touching the web. That decision point matters for brands, because it means Claude’s picture of you is a mix of two layers: what it learned in training, which you cannot edit, and what it retrieves live, which you can influence this quarter.
When it does search, Claude queries an external index, reads the top results, and since February 2026 filters fetched content with code, keeping the passages relevant to the question and discarding the rest. Then it writes the answer with inline citations, each one a clickable link to the source page. The citations are not decoration. They are the referral path from Claude’s answers to your site, and the visible record of which pages Claude trusted.
The evidence that the index is Brave
Anthropic has never published a press release naming its search provider, so the honest framing is an evidence chain, not an official fact. The chain is strong.
First, Anthropic’s own Trust Center lists Brave Search as a “Web Search” subprocessor, added on March 19, 2025, one day before web search launched, and scoped to all products. The listing was still live as of early July 2026. Second, within a day of launch, developer Simon Willison found a parameter named BraveSearchParams inside Claude’s search tool definition. Tools do not name parameters after vendors they do not call. Third, independent testers ran the same queries through Claude and through Brave and compared citations. The launch-day test found an 86.7 percent overlap, though that figure deserves a caveat it rarely gets: it was 13 matches out of 15 results across three queries, run by a vendor on day one. The same vendor re-ran the test in June 2026 on roughly 400 queries and got 79.2 percent. Still a strong signal, but not the near-perfect passthrough the original number implied.
Why Brave matters as context: it runs one of the few independent Western indexes at scale, over 40 billion pages built without leaning on Google or Bing. And the field around it has thinned. Microsoft retired the public Bing Search API in August 2025, and Google’s Custom Search API is closed to new customers and scheduled to shut down in January 2027. For AI companies that need a search backend, the independent options are few, which makes the Brave dependency more durable, not less.
The TurboPuffer wrinkle almost nobody covers
Here is the detail that unsettles the neat story. The same Anthropic subprocessor list that names Brave added a second “Web Search” entry on May 6, 2026: TurboPuffer, a vector and full-text search database, listed for all products except Claude for Government. TurboPuffer publicly lists Anthropic as a customer and says nothing about the use case. Anthropic classifies it as a web-search subprocessor and explains nothing further.
What that means is genuinely open. A retrieval database sitting next to a keyword index could cache and re-rank web content, could power semantic retrieval over candidate results, or could serve some adjacent feature entirely. No public source confirms which. What it does change is the assumption you can make: “Claude returns Brave’s top ten verbatim” can no longer be stated as settled. Architecturally, a keyword engine plus a retrieval layer is exactly the stack that could filter or reorder results beyond raw Brave rank. The practical takeaway for a brand is unchanged but firmer: optimize for being genuinely relevant and citable, not for gaming one index’s rank, because there may be a second selection step you cannot see.
The three bots, and why each one matters to you
Claude touches your website through three distinct crawlers, formally documented by Anthropic in February 2026, each with its own user-agent, its own job, and its own consequence if you block it.
ClaudeBot collects public web content that may be used to train future models. Claude-SearchBot crawls to improve the quality and relevance of Claude’s search results, the indexing layer. Claude-User fetches your page live at the moment a user’s question needs it, which makes it the citation moment: a Claude-User hit in your logs means a real answer pulled your page in. All three honor robots.txt, including Claude-User, which is worth underlining because OpenAI and Perplexity both warn that their user-initiated fetchers may not follow robots.txt the same way. Anthropic’s do.
The blocks are independent. Disallowing ClaudeBot keeps you out of future training data but does not touch search or live fetches. Disallowing Claude-SearchBot or Claude-User is what actually removes you from Claude’s answers, and plenty of sites have done it by accident with a blanket “block all AI bots” rule written in 2024. If Claude never cites you, your robots.txt is the first place to look. Two legacy user-agents, Claude-Web and Anthropic-AI, are deprecated, and Anthropic supports Crawl-delay if load is a concern. IP blocking is explicitly discouraged, because the bots run from public cloud ranges and blocking them can prevent the bots from reading your robots.txt at all.
What this means if you want Claude to cite you
Three practical consequences fall out of the mechanics. First, your Brave presence is your Claude presence, so check how your brand looks in Brave’s results, not only Google’s. Second, your pages must be fetchable: Claude-SearchBot and Claude-User allowed in robots.txt, content rendered server-side, because a page that needs JavaScript to display may as well be blank. Third, citations favor pages that answer the question directly with claims Claude can defend, which rewards answer-first structure over clever copy. The full playbook is in how to get your brand recommended by Claude, and you can see how Claude specifically ranks and describes brands in your category on the Claude visibility tracker page.
One more practical note: because Claude’s answers vary between runs and its index differs from Google’s, checking your Claude visibility by asking once tells you very little. Tracking the same buyer-intent prompts on a schedule is what shows your real position and its trend. Rankry does this for Claude alongside ChatGPT, Gemini, Perplexity, and Grok, keeping the raw Claude responses as evidence.
FAQ
What search engine does Claude use? Anthropic has never officially named one, but the public evidence points strongly at Brave Search: Brave sits on Anthropic’s subprocessor list for all products, Claude’s tool definition exposed Brave-specific parameters, and independent tests show high citation overlap with Brave’s results.
When does Claude search the web? Claude decides on its own, per question. If the answer needs information past its training data, like current prices, recent events, or live availability, it runs a search. Questions it can answer from training knowledge, it answers without searching.
Does Claude cite its sources? Yes. Every web-sourced claim carries an inline citation linking to the page it came from, in the chat product and in the API. Those citations are also a referral traffic path to the cited sites.
What are ClaudeBot, Claude-SearchBot, and Claude-User? Anthropic’s three crawlers. ClaudeBot collects content for model training, Claude-SearchBot indexes content for Claude’s search quality, and Claude-User fetches pages live when a user’s question needs them. All three respect robots.txt, and blocking one does not block the others.
Why does Claude cite my competitor but not me? Usually one of four gates: you are weak in the index Claude searches, your pages are blocked or unreadable to its bots, your content does not directly answer the question, or independent sources corroborate your competitor and not you. Diagnose in that order.
See exactly how Claude ranks, describes, and cites brands in your category, tracked weekly with the raw responses kept as evidence. Start a free 7-day Rankry trial, no card, first report in two minutes.