Comment on Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives
BrikoX@lemmy.zip 4 days ago
Ignoring the self-promo part of the article, Cloudflare is in big part responsible for all of this. Their home page promotes mostly “AI” tools, they are even offering “pay-per-crawl” to make it a viable business model.
So while you correctly shit on Perplexity, don’t forget that Cloudflare is even worse enemy and not a friend.