Comment on Microsoft and Reddit Are Fighting About Why Bing’s Crawler Is Blocked on Reddit
theangriestbird@beehaw.org 2 months ago
The beef between Microsoft and Reddit came to light after I published a story revealing that Reddit is currently blocking every crawler from every search engine except Google, which earlier this year agreed to pay Reddit $60 million a year to scrap the site for its generative AI products.
I know the author meant “scrape”, but sometimes it really does feel like AI is just scrapping the old internet for parts.
cybermass@lemmy.ca 2 months ago
Yeah, aren’t like over half of reddit comments/posts by bots these days?
originalucifer@moist.catsweat.com 2 months ago
yep, and the longer that happens the less value to the dataset. its becoming aged.
RiikkaTheIcePrincess@pawb.social 2 months ago
[Joke] See, Reddit’s doing a good thing here! They’re making sure nobody ends up toxifying their own dataset by using Reddit’s garbage heap of bot posts!
originalucifer@moist.catsweat.com 2 months ago
google needs a checkbox of 'ignore reddit' im sick of having to manually add -reddit