Comment on Wikipedia is giving AI developers its data to fend off bot scrapers
p03locke@lemmy.dbzer0.com 6 days agoIt’s a common problem. People writing bot scrapers for public data, which costs a lot of bandwidth, when they could have easily just downloaded the entire dataset from a dedicated link. Finding better ways to tell them “Hey, morons, go download the goddamn link!” saves on that bandwidth and web server CPU.