Cloudflare to Block Google Crawlers by Default on Sept. 15 as AI Search Cuts Publisher Traffic
Updated
Updated · The Ringer · Aug 4
Cloudflare to Block Google Crawlers by Default on Sept. 15 as AI Search Cuts Publisher Traffic
3 articles · Updated · The Ringer · Aug 4
Summary
Sept. 15 is when Cloudflare will switch its settings to block Google crawlers by default, a notable escalation in publishers’ pushback against AI-driven search.
Google’s AI Overviews and AI Mode are keeping users on Google pages instead of sending them outward, leaving sites with less referral traffic while still exposing their content to AI scraping.
75% is how much traffic All About Berlin says it has lost since AI Overviews launched in 2024; SparkToro says 68% of U.S. Google searches now end without a click, up from 45% a decade ago.
Publishers are increasingly weighing broader withdrawal: USA Today says it may block Google, People Inc. says doing so is '100% on the table,' and Reddit has reportedly reconsidered renewing its AI licensing deal.
57% of web requests now come from automated bots, Cloudflare researchers said, underscoring a wider shift from an internet built around human clicks to one increasingly dominated by AI agents.