Cloudflare has introduced a new policy mandating AI firms to clearly identify and differentiate their web crawlers used for search engines from those deployed for AI training and agents by September 15.
If companies fail to comply, Cloudflare will block these AI training crawlers by default across a broad range of publisher websites. The measure aims to protect publishers' content from unauthorized AI usage and ensure proper compensation or control when their data is utilized for AI training.











