By Iliass El Barhoumi · Published · Last updated
What is Amazonbot?
Amazonbot is Amazon’s web crawler. Amazon says it is used to improve its products and services, helping provide more accurate information to customers, and that the content may be used to train Amazon AI models. Amazon runs search under a separate token, Amzn-SearchBot, and user-requested fetches under Amzn-User.
| Operator | Amazon |
|---|---|
| Purpose | training · Alexa and Amazon’s AI models |
| robots.txt token | Amazonbot |
| In the user-agent | compatible; Amazonbot/0.1 |
| CrawlRadar weight | 1 of the crawler-access weights, never flagged critical |
| Vendor docs | Amazon: About Amazonbot |
How to allow Amazonbot in robots.txt
Give Amazonbot its own group with Allow: /. A named group overrides the User-agent: * group, so this works even when the wildcard disallows everything:
User-agent: Amazonbot
Allow: /How to block Amazonbot in robots.txt
Replace Allow with Disallow. To block only part of the site, disallow that path instead of /:
User-agent: Amazonbot
Disallow: /- Tokens are matched case-insensitively, but writing
Amazonbotexactly as the vendor spells it avoids surprises with stricter parsers. - Check your CDN and security plugins too: a firewall rule can block Amazonbot without any line in robots.txt.
Should you block Amazonbot?
Allow Amazonbot if Alexa and Amazon’s assistants are a channel your customers use, and block it if you want out of Amazon’s model training. Amazon documents each of its tokens as independent, so blocking Amazonbot does not block Amzn-SearchBot, and says changes take about 24 hours to apply.
How to verify a request is really Amazonbot
Run a reverse DNS lookup on the requesting IP: a genuine Amazonbot host resolves under crawl.amazonbot.amazon, and a forward lookup on that hostname returns the same IP. Amazon also publishes its addresses at https://developer.amazon.com/amazonbot/ip-addresses/. CrawlRadar’s collector runs this check on every hit.
Is Amazonbot visiting your site?
Your analytics will not tell you. AI crawlers do not run JavaScript, so a tag-based tool such as Google Analytics never records them. CrawlRadar’s collector reads your own request path and records each Amazonbot visit, the page it fetched, and whether it came from Amazon. See the full AI crawler directory for the other tokens CrawlRadar checks.
Frequently asked questions
How do I verify that a request is really Amazonbot?
Amazon publishes Amazonbot’s IP addresses at developer.amazon.com/amazonbot/ip-addresses, and genuine hosts resolve under crawl.amazonbot.amazon on a reverse DNS lookup. CrawlRadar’s collector runs the reverse-DNS check on every hit.
Does blocking Amazonbot affect my Amazon seller listings?
No. The rule controls crawling of your own website. Listings on Amazon are not fetched from your site by Amazonbot.