(Editor’s Note: The Technoskeptic has no paid relationship with Cloudflare. We don’t pay for their services, and they don’t pay us-or give us free commercial services-for any endorsements. We simply appreciate their “free for ordinary consumer” tools.)
It is a new fact of life that large AI companies have bots constantly scraping the internet for training data to feed into the maws of their AI models. The AI companies use the scraped information for commercial purposes-but take it without the consent of, or compensation to, those who created it. In January, we told you about Glaze and Nightshade, two free computer tools that visual artists can use to protect their work from AI scrapers by “poisoning” the dataset of the AI bots.
For the 4th of July, we thought it would be useful to tell readers about a new bot privacy shield built into Cloudflare’s free service. Cloudflare protects a large chunk of the internet from various kinds of malicious attacks, most often the Distributed Denial of Service (DDOS) attacks that flood websites with garbage to make them unreachable.
A large chunk of what Cloudflare does, it does for free. (1)
It probably won’t surprise you to learn that although there is a notice, the creation of a “robot.txt” file on a website, that tells web crawlers, “This site has ‘opted-out’ of being scraped,” many online companies-particularly AI companies-do NOT respect the assertion of privacy made by the “robot.txt” file.
In steps Cloudflare to fight those web-crawling scofflaws! Fair warning: getting this protection does require signing your website up for Cloudlflare’s free service, which includes several features that make your site faster and safer. The latest freebie is Bot Fight Mode. Once activated, Bot Fight Mode will block dozens of AI bots from getting to your content. It doesn’t rely on AI crawlers and scrapers to politely respect the wishes of site owners as expressed by “robot.txt” files. When Bot Fight Mode is on, excluded bots cannot get to your site at all, especially the bots that try to hide that they are bots, i.e. the worst bots.
If you have online content you don’t want those plundering AI bots to steal, Coudflare’s Bot Fight Mode can protect your website from content agglomeration without compensation-a great way to declare your AI independence on the 4th of July!
I’ve used Cloudflare’s 1.1.1.1 free app for years. It keeps your internet browsing anonymous and doesn’t sell user information.