Comment on A big announcement from Skill Up (new website)
HeyJoe@lemmy.world 10 months agoI don’t blame them, I also think the pay wall stops AI from stealing everything as well. I just wish they had a $5 tier. It seemed to be one of the big comments on the video, and I agree with it. Sure its less, but I would imagine the extra people gained from that tier would certainly help since its lost money at the moment.
pirateKaiser@sh.itjust.works 10 months ago
As someone who works for a paywallled website, that’s hardly a deterrent. If the site is important enough, they will pay for accounts and crawl until the server melts
HeyJoe@lemmy.world 10 months ago
Is there any true way to block it? Does the crawler literally use the same access (443) as us to scrape content? If so, the only other thing I can think of is to block all known IP’s that AI crap originates from, but that sounds daunting and impossible to catch everything.
pirateKaiser@sh.itjust.works 10 months ago
There’s no fullproof way. Even if you somehow block every crawling automation, there’s still puppeteering where the bot behaves just like a normal user.