Comment on A big announcement from Skill Up (new website)
HeyJoe@lemmy.world 2 days agoI don’t blame them, I also think the pay wall stops AI from stealing everything as well. I just wish they had a $5 tier. It seemed to be one of the big comments on the video, and I agree with it. Sure its less, but I would imagine the extra people gained from that tier would certainly help since its lost money at the moment.
pirateKaiser@sh.itjust.works 1 day ago
As someone who works for a paywallled website, that’s hardly a deterrent. If the site is important enough, they will pay for accounts and crawl until the server melts
HeyJoe@lemmy.world 1 day ago
Is there any true way to block it? Does the crawler literally use the same access (443) as us to scrape content? If so, the only other thing I can think of is to block all known IP’s that AI crap originates from, but that sounds daunting and impossible to catch everything.
pirateKaiser@sh.itjust.works 14 hours ago
There’s no fullproof way. Even if you somehow block every crawling automation, there’s still puppeteering where the bot behaves just like a normal user.