![Anthropic logo](/vendors/anthropic.svg)

Anthropic · live answer fetch

# Is Claude-User blocked on your site?

Claude-User is Anthropic’s live answer fetch token for Claude conversations, and whether it reaches you is decided by one line in your robots.txt. It fetches your page when someone asks Claude a question that needs it. Blocking it means Claude cannot retrieve your page for that person.

[Check my robots.txt](/tools/ai-crawler-checker)[Or score the whole page →](/tools/geo-score-checker)

- Not critical if blocked
- Claude-User robots.txt token

By [Iliass El Barhoumi](/about) · Published 1 October 2026 · Last updated 1 October 2026

## What is Claude-User?

Claude-User is the user agent Claude sends when a person asks a question that needs a live web page, or shares a link for Claude to read. Anthropic describes it as supporting Claude users directly: each request comes from a conversation, not from a scheduled crawl.

| Operator          | Anthropic                                                                                                                                                                             |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Purpose           | live answer fetch · Claude conversations                                                                                                                                              |
| robots.txt token  | Claude-User                                                                                                                                                                           |
| In the user-agent | Claude-User                                                                                                                                                                           |
| CrawlRadar weight | 1 of the crawler-access weights, never flagged critical                                                                                                                               |
| Vendor docs       | [Anthropic: Does Anthropic crawl data from the web?](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler) |

## How to allow Claude-User in robots.txt

Give Claude-User its own group with `Allow: /`. A named group overrides the `User-agent: *` group, so this works even when the wildcard disallows everything:

```
User-agent: Claude-User
Allow: /
```

## How to block Claude-User in robots.txt

Replace `Allow` with `Disallow`. To block only part of the site, disallow that path instead of `/`:

```
User-agent: Claude-User
Disallow: /
```

- Tokens are matched case-insensitively, but writing `Claude-User` exactly as the vendor spells it avoids surprises with stricter parsers.
- Check your CDN and security plugins too: a firewall rule can block Claude-User without any line in robots.txt.

## Should you block Claude-User?

Leave Claude-User allowed. Anthropic’s documentation says disabling it prevents Claude from retrieving your content in response to a user’s query, which may reduce your visibility for user-directed web search. A block means Claude cannot open your page even when someone asks about it by name.

## How to verify a request is really Claude-User

Anthropic publishes the IP addresses Claude-User uses at https://claude.com/crawling/bots.json. A request that claims to be Claude-User from any other address is someone borrowing the name, and should be treated like any other scraper. CrawlRadar’s collector checks every hit against that list and labels it verified or spoofed.

## Anthropic’s other AI crawlers

Each Anthropic token is a separate robots.txt decision. Blocking one has no effect on the others.

| Token                                          | Purpose      | What blocking it costs                                                                                                                           |
| ---------------------------------------------- | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| [ClaudeBot](/crawlers/claudebot)               | training     | Blocking it keeps your future pages out of Anthropic’s training data. Claude’s search citations come from Claude-SearchBot, which is unaffected. |
| [Claude-SearchBot](/crawlers/claude-searchbot) | search index | This is the crawler that indexes pages for Claude’s search answers. Blocking it can remove you from the results Claude cites.                    |
| [anthropic-ai](/crawlers/anthropic-ai)         | training     | A legacy Anthropic token. Worth keeping in your robots.txt alongside ClaudeBot so an older group cannot contradict a newer one.                  |

## Is Claude-User visiting your site?

Your analytics will not tell you. AI crawlers do not run JavaScript, so a tag-based tool such as Google Analytics never records them. CrawlRadar’s collector reads your own request path and records each Claude-User visit, the page it fetched, and whether it came from Anthropic. To check by hand, the guide to [tracking AI crawlers](/blog/how-to-track-ai-crawlers) has the log commands and each vendor’s IP lists. See the [full AI crawler directory](/crawlers) for the other tokens CrawlRadar checks.

## Frequently asked questions

- ### Does Claude-User follow robots.txt?  
Yes. Anthropic documents Claude-User as a robots.txt token site owners can use to control which sites user-initiated requests can reach, and says its bots honour robots.txt directives.
- ### Will Claude-User visits show up in my analytics?  
No. Like every AI fetcher it does not run JavaScript, so an analytics tag never fires. It shows up in server logs and in CrawlRadar’s collector, which reports it separately from Claude-SearchBot and ClaudeBot.

## See whether Claude-User actually visits you

robots.txt says who may crawl; only your own request log says who did. CrawlRadar’s collector records every AI crawler visit, verifies it against the vendor’s published ranges, and shows the pages each one fetched.

Or start with the free [AI crawler checker](/tools/ai-crawler-checker).

---

Source: https://www.crawlradar.xyz/crawlers/claude-user
