![Anthropic logo](/vendors/anthropic.svg)

Anthropic · training

# Is ClaudeBot blocked on your site?

ClaudeBot is Anthropic’s training token for Anthropic model training, and whether it reaches you is decided by one line in your robots.txt. Blocking it keeps your future pages out of Anthropic’s training data. Claude’s search citations come from Claude-SearchBot, which is unaffected.

[Check my robots.txt](/tools/ai-crawler-checker)[Or score the whole page →](/tools/geo-score-checker)

- Not critical if blocked
- ClaudeBot robots.txt token

By [Iliass El Barhoumi](/about) · Published 1 October 2026 · Last updated 1 October 2026

## What is ClaudeBot?

ClaudeBot is the crawler Anthropic uses to collect public web content that could contribute to training its models. Anthropic documents it alongside Claude-SearchBot, which indexes pages for search, and Claude-User, which fetches a page when a user asks Claude a question. Restricting ClaudeBot signals that your future content should be excluded from training.

| Operator          | Anthropic                                                                                                                                                                             |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Purpose           | training · Anthropic model training                                                                                                                                                   |
| robots.txt token  | ClaudeBot                                                                                                                                                                             |
| In the user-agent | compatible; ClaudeBot/1.0; +claudebot@anthropic.com                                                                                                                                   |
| CrawlRadar weight | 1 of the crawler-access weights, never flagged critical                                                                                                                               |
| Vendor docs       | [Anthropic: Does Anthropic crawl data from the web?](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler) |

## How to allow ClaudeBot in robots.txt

Give ClaudeBot its own group with `Allow: /`. A named group overrides the `User-agent: *` group, so this works even when the wildcard disallows everything:

```
User-agent: ClaudeBot
Allow: /
```

## How to block ClaudeBot in robots.txt

Replace `Allow` with `Disallow`. To block only part of the site, disallow that path instead of `/`:

```
User-agent: ClaudeBot
Disallow: /
```

- Tokens are matched case-insensitively, but writing `ClaudeBot` exactly as the vendor spells it avoids surprises with stricter parsers.
- Check your CDN and security plugins too: a firewall rule can block ClaudeBot without any line in robots.txt.

## Should you block ClaudeBot?

Block ClaudeBot to keep your pages out of Anthropic’s training data; it does not remove you from Claude’s search answers, which are governed by Claude-SearchBot. If the concern is server load rather than training, Anthropic says its bots support the non-standard Crawl-delay directive, so you can slow ClaudeBot down instead of blocking it.

## How to verify a request is really ClaudeBot

Anthropic publishes the IP addresses ClaudeBot uses at https://claude.com/crawling/bots.json. A request that claims to be ClaudeBot from any other address is someone borrowing the name, and should be treated like any other scraper. CrawlRadar’s collector checks every hit against that list and labels it verified or spoofed.

## Anthropic’s other AI crawlers

Each Anthropic token is a separate robots.txt decision. Blocking one has no effect on the others.

| Token                                          | Purpose           | What blocking it costs                                                                                                                      |
| ---------------------------------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
| [Claude-SearchBot](/crawlers/claude-searchbot) | search index      | This is the crawler that indexes pages for Claude’s search answers. Blocking it can remove you from the results Claude cites.               |
| [Claude-User](/crawlers/claude-user)           | live answer fetch | It fetches your page when someone asks Claude a question that needs it. Blocking it means Claude cannot retrieve your page for that person. |
| [anthropic-ai](/crawlers/anthropic-ai)         | training          | A legacy Anthropic token. Worth keeping in your robots.txt alongside ClaudeBot so an older group cannot contradict a newer one.             |

## Is ClaudeBot visiting your site?

Your analytics will not tell you. AI crawlers do not run JavaScript, so a tag-based tool such as Google Analytics never records them. CrawlRadar’s collector reads your own request path and records each ClaudeBot visit, the page it fetched, and whether it came from Anthropic. To check by hand, the guide to [tracking AI crawlers](/blog/how-to-track-ai-crawlers) has the log commands and each vendor’s IP lists. See the [full AI crawler directory](/crawlers) for the other tokens CrawlRadar checks.

## Frequently asked questions

- ### Does blocking ClaudeBot stop Claude from citing my site?  
No. Claude’s search answers draw on pages indexed by Claude-SearchBot, and live fetches come from Claude-User. ClaudeBot only governs whether your content may be used for training.
- ### What is the difference between ClaudeBot and anthropic-ai?  
anthropic-ai is an older token. ClaudeBot, Claude-SearchBot and Claude-User are the three Anthropic documents today. Listing anthropic-ai as well is harmless as long as its rule matches your ClaudeBot rule.

## See whether ClaudeBot actually visits you

robots.txt says who may crawl; only your own request log says who did. CrawlRadar’s collector records every AI crawler visit, verifies it against the vendor’s published ranges, and shows the pages each one fetched.

Or start with the free [AI crawler checker](/tools/ai-crawler-checker).

---

Source: https://www.crawlradar.xyz/crawlers/claudebot
