![Anthropic logo](/vendors/anthropic.svg)

Anthropic · training

# Is anthropic-ai blocked on your site?

anthropic-ai is Anthropic’s training token for Anthropic (legacy), and whether it reaches you is decided by one line in your robots.txt. A legacy Anthropic token. Worth keeping in your robots.txt alongside ClaudeBot so an older group cannot contradict a newer one.

[Check my robots.txt](/tools/ai-crawler-checker)[Or score the whole page →](/tools/geo-score-checker)

- Not critical if blocked
- anthropic-ai robots.txt token

By [Iliass El Barhoumi](/about) · Published 1 October 2026 · Last updated 1 October 2026

## What is anthropic-ai?

anthropic-ai is an older robots.txt token associated with Anthropic’s data collection. Anthropic’s current crawler documentation names ClaudeBot, Claude-SearchBot and Claude-User instead, and live requests that identify themselves as anthropic-ai are rare.

| Operator          | Anthropic                                                                                                                                                                             |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Purpose           | training · Anthropic (legacy)                                                                                                                                                         |
| robots.txt token  | anthropic-ai                                                                                                                                                                          |
| In the user-agent | Never sent: this token is only read from robots.txt                                                                                                                                   |
| CrawlRadar weight | 1 of the crawler-access weights, never flagged critical                                                                                                                               |
| Vendor docs       | [Anthropic: Does Anthropic crawl data from the web?](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler) |

## How to allow anthropic-ai in robots.txt

Give anthropic-ai its own group with `Allow: /`. A named group overrides the `User-agent: *` group, so this works even when the wildcard disallows everything:

```
User-agent: anthropic-ai
Allow: /
```

## How to block anthropic-ai in robots.txt

Replace `Allow` with `Disallow`. To block only part of the site, disallow that path instead of `/`:

```
User-agent: anthropic-ai
Disallow: /
```

- Tokens are matched case-insensitively, but writing `anthropic-ai` exactly as the vendor spells it avoids surprises with stricter parsers.
- Check your CDN and security plugins too: a firewall rule can block anthropic-ai without any line in robots.txt.

## Should you block anthropic-ai?

Keep its rule identical to your ClaudeBot rule. A file that disallows anthropic-ai but allows ClaudeBot sends a mixed signal nobody intended, and many robots.txt block-list generators still add it. Removing it changes nothing for Anthropic’s current crawlers.

## How to verify a request is really anthropic-ai

There is no reliable way: Anthropic publishes no IP ranges for this retired token. A request claiming to be anthropic-ai could be anyone, so CrawlRadar records such hits as unverified rather than crediting them to Anthropic.

## Anthropic’s other AI crawlers

Each Anthropic token is a separate robots.txt decision. Blocking one has no effect on the others.

| Token                                          | Purpose           | What blocking it costs                                                                                                                           |
| ---------------------------------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| [ClaudeBot](/crawlers/claudebot)               | training          | Blocking it keeps your future pages out of Anthropic’s training data. Claude’s search citations come from Claude-SearchBot, which is unaffected. |
| [Claude-SearchBot](/crawlers/claude-searchbot) | search index      | This is the crawler that indexes pages for Claude’s search answers. Blocking it can remove you from the results Claude cites.                    |
| [Claude-User](/crawlers/claude-user)           | live answer fetch | It fetches your page when someone asks Claude a question that needs it. Blocking it means Claude cannot retrieve your page for that person.      |

## Is anthropic-ai visiting your site?

Your analytics will not tell you. AI crawlers do not run JavaScript, so a tag-based tool such as Google Analytics never records them. CrawlRadar’s collector reads your own request path and records each anthropic-ai visit, the page it fetched, and whether it came from Anthropic. To check by hand, the guide to [tracking AI crawlers](/blog/how-to-track-ai-crawlers) has the log commands and each vendor’s IP lists. See the [full AI crawler directory](/crawlers) for the other tokens CrawlRadar checks.

## Frequently asked questions

- ### Do I still need anthropic-ai in robots.txt?  
Not for Anthropic’s current crawlers, which identify as ClaudeBot, Claude-SearchBot and Claude-User. Leaving it in is harmless as long as its rule matches your ClaudeBot rule.
- ### Is a request claiming to be anthropic-ai genuine?  
Probably not. There are no published IP ranges to check it against, so CrawlRadar records such hits as unverified rather than crediting them to Anthropic.

## See whether anthropic-ai actually visits you

robots.txt says who may crawl; only your own request log says who did. CrawlRadar’s collector records every AI crawler visit, verifies it against the vendor’s published ranges, and shows the pages each one fetched.

Or start with the free [AI crawler checker](/tools/ai-crawler-checker).

---

Source: https://www.crawlradar.xyz/crawlers/anthropic-ai
