AI bot & crawler traffic
Copy linkAI bots scan public websites to collect data used in search engines, AI model training, and other data processing tasks
Latest AI blog posts
Copy linkPosts tagged with "AI" from the Cloudflare blog
Precursor, our new continuous behavioral validation engine for bot management, offers visibility into how humans and bots actually interact across the full user journey. By turning session-level behavior into bot detection signals, it identifies advanced automation with higher precision — while reducing friction for legitimate users.
Marina Elmore, Benedikt Wolters
We're opening the waitlist for our Monetization Gateway, which will allow you to charge for any web page, dataset, API, or MCP tool behind Cloudflare. The charges will settle in stablecoins over the x402 open protocol, with no payments stack of your own to build.
Rohin Lohe, Justin Ridgely, Will Papper
One year after declaring Content Independence Day, a dynamic market for monetized content has officially emerged. In this report, we examine how the rise of autonomous AI agents is upending traditional search referrals and detail the new infrastructure required to support a sustainable web economy.
Arielle Weiss, Zach Albertson, Emily Lanfear