Cloudflare’s new policy pushes AI companies to pay for publishers’ content

Cloudflare has introduced a policy requiring AI developers to clearly distinguish between bots used for search indexing and those used to train artificial intelligence models by September 15. Publishers who use the company's services will now be able to block AI scrapers by default while still allowing search engines to crawl their websites. This change provides content creators with more granular control over how their data is used, addressing long-standing concerns regarding unauthorized scraping for generative AI datasets.