CPMs are better if traffic is not bots. Same at my company we have to use cloudflare. They essentially don’t know who is human and who is OpenClaw anymore or scraper.
Comment on this absolute bullshit from reddit
mic_check_one_two@lemmy.dbzer0.com 1 day agoMy only guess is because they want to force AI scrapers to pay for access. And most scrapers probably work best on old.reddit because they keep changing new Reddit’s layout. As a programmer, if you’re going to code something to access a site, (and you don’t want to constantly have to rework it every time they make a small UI update) you’re going to use the version that never changes. So by forcing them to log in, it makes tracking the individual users (and thus, identifying potential unauthorized scraper accounts) easier.
laurenceOfSuburbia@lemmy.world 19 hours ago
AlteredEgo@lemmy.ml 19 hours ago
But can’t you use very simple agentic AI to constantly update the code for web scrapers on new reddit? From what I understand that is the kind of coding LLM can do well and you can just unit test for errors and feed it back to the LLM until it works. Even requiring accounts, creation can be automated using LLMs too.
This sounds like it a loosing proposition and will only piss off hardcore users. Not that this would stop them. But I don’t think protection from web scrapers is a good explanation. Probably it’s just enshittification, forcing users to use their new interface to optimize business metrics. Someone is going to get a bonus because they improved the short sighted metrics.
irelephant@lemmy.dbzer0.com 16 hours ago
New reddit is really weird, it’s all html (even pagination/apis) with no clear consistent structure.
The mobile apis are easier to scrape