Yes, if Claude-SearchBot can reach you. Anthropic runs three crawlers and restructured them in February 2026. Unlike Perplexity, Anthropic publishes no crawler IP ranges and states that IP blocking may not work, so control access through robots.txt per crawler. Claude carries about 3 percent of assistant fetches, weighted toward technical and professional buyers rather than consumers.
Anthropic tells site owners something the other engines do not. It publishes no IP ranges for its crawlers, and its own help centre warns that blocking by IP “may not work correctly”.
Perplexity says the opposite. It publishes address ranges and instructs you to match user-agent against IP before acting. Two engines, two contradictory pieces of advice, and a South African firm reading a generic guide gets whichever one the author happened to copy.
This article covers what Anthropic documents, what independent measurement shows, and why Claude’s small share deserves more attention from B2B firms than the number suggests.
Three crawlers, renamed in February 2026
Anthropic restructured its crawler naming around 20 February 2026, replacing broader earlier descriptions with three named agents.
| User-agent | What Anthropic says it does | Honours robots.txt |
|---|---|---|
ClaudeBot | Crawls the web to gather model training data | Yes |
Claude-SearchBot | Indexes web content to power Claude’s search feature | Yes |
Claude-User | Fetches a page live when a person asks Claude about it or pastes the URL | Yes |
Claude-SearchBot governs whether Claude can surface you in a search-style answer. That is the one to allow if you want visibility. ClaudeBot is a separate decision about training, and you can allow one while blocking the other.
Two older names, Claude-Web and Anthropic-AI, appear in third-party trackers as deprecated predecessors. Anthropic’s current documentation does not list them, so rules referencing those strings are doing nothing.
Blocking uses ordinary syntax, repeated per subdomain:
User-agent: ClaudeBot
Disallow: /Anthropic supports the non-standard Crawl-delay directive, which most engines ignore.
The IP verification difference
Anthropic publishes no IP ranges for these three crawlers, because they run on shared service-provider addresses, and it discourages IP blocking as unreliable.
One caution against confusion. Anthropic does publish a fixed IP list, at its API documentation, covering traffic from the Claude API and its connectors. Those addresses have nothing to do with the three web crawlers. Firewall rules built on that list will not catch ClaudeBot.
For Claude you therefore verify by name and accept that a spoofed user-agent is indistinguishable. That is a weaker position than Perplexity offers, and Anthropic says so rather than pretending otherwise.
What Claude’s search runs on
Anthropic does not say.
Its consumer help documentation describes Claude invoking a search tool without naming a provider for text results. It does confirm that images in search results come from Bing.
Brave gets named in one place: the web search connector for Claude for Government, where Anthropic documents sending only the query string to the Brave Search API. Writers have generalised that into “Claude uses Brave”, which overstates a document about a government product. Treat the provider behind mainstream Claude as undocumented.
Citations are documented and dependable. The API web search tool returns each result with a URL, title and page age, and each citation carries the source URL, title, and up to 150 characters of the cited text. Consumer Claude shows source links to users.
Source ranking is not documented anywhere. Anthropic publishes nothing on how it chooses between candidates, so we test rather than theorise.
Three surfaces, and only one of them is a crawler
Business owners collapse these into “Claude read my site”. The three behave apart from each other.
Consumer search inside Claude.ai runs on Claude-SearchBot indexing plus live fetches. This is the surface where visibility work applies.
The web fetch tool retrieves a specific URL already in the conversation, from a search result or a pasted link. It discovers nothing new, and Anthropic documents it as respecting robots.txt, returning a url_not_allowed error when blocked.
Claude for Chrome takes a third route. The extension reads pages through screenshots inside the user’s own logged-in browser session. Your server sees a single authenticated page load from a real person’s browser, not a bot pattern. Nothing in Anthropic’s documentation says the extension consults robots.txt, and nothing says it does not.
That last surface matters for a practical reason. If a client uses Claude for Chrome to work through your quotation page, your analytics record an ordinary visit. You will not attribute it to Claude.
Claude is small, and that is the wrong reason to ignore it
A twelve-week server-log study across 125 sites measured user-triggered assistant fetches. ChatGPT took close to 96 percent. Claude took 3.2 percent. Perplexity took under 1 percent.
By crawl volume across the same panel, Anthropic’s crawlers and fetchers produced around 76,000 hits against OpenAI’s 213,000 and Googlebot’s 444,000.
On those numbers Claude looks skippable. Then look at who is asking.
Anthropic’s own Economic Index reports that coding and technical work dominates Claude usage, at 34 to 37 percent of consumer conversations and 44 to 52 percent of first-party API traffic. Business API traffic runs 77 percent toward full task delegation, against about 50 percent on the consumer product.
Read that as a buyer profile. Claude’s users skew toward developers, engineers, analysts and professional services. A Johannesburg software firm, an engineering consultancy or a technical supplier meets a different mix of buyers inside Claude than inside a general assistant. A retail bakery does not.
South Africa appears on Anthropic’s supported countries list for both Claude and the API, and AWS confirmed in January 2026 that Claude models are reachable from its Cape Town region. Access is not the constraint.
The settlement, and what it does not mean for you
A court gave final approval on 20 July 2026 to a 1.5 billion dollar copyright settlement against Anthropic, covering training on books obtained through pirated libraries. Around 500,000 works, at 3,000 dollars each.
Read the ruling before drawing conclusions. The court held that training on books Anthropic obtained through legal channels was fair use. The infringement concerned acquiring and retaining pirated copies.
Nothing in that outcome gives your business a claim, a payment or a new control. Any provider citing the settlement as leverage over Anthropic on your behalf is selling you a story.
News Corp’s chief executive told an earnings call in May 2026 that the company anticipates a licensing arrangement with Anthropic. Anthropic has not confirmed a figure. No programme exists for a small business to license content, submit a site, or pay for inclusion.
What to do this week
Check your robots.txt for Claude-SearchBot. A blanket Disallow under User-agent: * catches it, and that single line is the difference between eligible and invisible.
Decide ClaudeBot on its own. Training access and search access are different questions, and your answers need not match.
Then ask Claude ten questions your buyers would ask, in your category, and record which businesses it names and which sources it cites. Given Claude’s technical skew, phrase them the way a professional buyer would rather than a consumer.
Read your logs for all three agents. Accept that you cannot verify them by address, because Anthropic says you cannot.
If your buyers are technical, weight this work higher than Claude’s 3 percent suggests. If they are not, do the robots.txt line and spend your budget elsewhere.
What we cannot tell you
Anthropic publishes no South African usage figures, and its Economic Index does not break out African countries in the material we reviewed. The share numbers in this article come from one independent server-log panel of 125 sites, whose authors disclose that it skews toward small business sites already seeking an AI visibility tool.
We cannot tell you what search provider sits behind mainstream Claude, how Claude ranks sources, or whether Claude for Chrome honours robots.txt. Anthropic documents none of it.
On llms.txt, the same twelve-week study recorded ClaudeBot fetching the file 9 times across 83 sites, against 67 fetches by Googlebot and 193 by Meta’s crawler. Nine fetches over three months is not a channel.
Where this leaves you
Claude gives you one clear lever, one honest limitation, and a buyer profile worth checking against your own.
Allow Claude-SearchBot. Accept that you cannot verify it by IP. Then decide whether your buyers are the kind of people who reach for Claude, because that question decides how much this deserves.
If you want the baseline run across Claude and seven other engines with the raw results handed over, our AI search visibility service sets out the scope, the price, and what we cannot promise.