AI bot & crawler traffic
Copy linkAI bots scan public websites to collect data used in search engines, AI model training, and other data processing tasks
Latest AI blog posts
Copy linkPosts tagged with "AI" from the Cloudflare blog
AI Search makes search easier than ever, with no Cloudflare primitives to stitch together. Point it at your data to create a search for your own files and websites. We're also sharing a preview of our new pricing model.
Nelson Duarte, Tiago Teles, Anni Wang
The next version of MCP has a rewritten, stateless core that just works on Workers. We cover upgrades to the protocol, the new feature lifecycle and SDK migration path, and hear from early adopters already running it in production.
Matt Carey
More than half of requests now come from machines, not people. Agent Readiness shows how well agents can discover and read your site, while Answer Engine Optimization tracks how often AI assistants recommend you.
Matthew Conroy, Jack Galilee