ClaudeBot
Anthropic's web crawler that indexes content for Claude's training data and retrieval-augmented generation.
Overview
- Operated By
- Anthropic
- Purpose
- Gathers web content for Claude model training and real-time web search capabilities.
- User-Agent String
- ClaudeBot/1.0
- Respects robots.txt
- Yes
- Category
- ai training
- Website
- https://anthropic.com