AI agent directory · Anthropic
Claude-SearchBot
Claude-SearchBot is Anthropic's crawler for building the index its search and answers draw on. Here's how to recognize it and check a request really is it.
At a glance
Claude-SearchBot facts
- Operator: Anthropic
- Type: Crawler
- Why it fetches: Builds a search or answer index
- User agent contains: `Claude-SearchBot`
- Verified by: published IP ranges
- Operator's documentation: support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler
Is it really?
How to tell a real Claude-SearchBot from an impostor
Anyone can put "Claude-SearchBot" in a user agent. What can't be faked is below.
- Check the address. Anthropic publishes the IP ranges Claude-SearchBot uses: claude.com/crawling/bots.json. A request whose user agent says Claude-SearchBot from an address outside them is an impostor.
In Punchline
How Punchline counts Claude-SearchBot
Punchline labels a request that names Claude-SearchBot with how sure it is: verified or declared when the evidence checks out, a lower tier when it can't be checked, and impostor when the evidence contradicts it. Each visit is counted under Anthropic, with its purpose, so you can see whether Claude-SearchBot is reading your site and which pages it reads. How it works · Known agents