Less sites are doing that to intentionally avoid tipping off the AIs about thresholds and trigger behaviours. The two main solutions are browser crypto challenges like this, or silently diverting flagged connections to a tar-pit of fake pages. The former is easier to implement but the latter actively sabotages the scraped data.
Comment on Tried to login to the arch wiki but was given a 3 minute crypto challenge.
PumaStoleMyBluff@lemmy.world 6 hours agoIt would be better to serve a static page saying “the server is getting hammered really hard”, and, if possible, context about the duration of the current event and historical frequency.
Funwayguy@lemmy.world 1 hour ago
kuhli@lemmy.dbzer0.com 4 hours ago
Disagree, that just stops me from using it completely instead of just having to wait a bit