AI crawlers need to be more respectful

Submitted ⁨⁨1⁩ ⁨year⁩ ago⁩ by ⁨bot@lemmy.smeargle.fans [bot]⁩ to ⁨hackernews@lemmy.smeargle.fans⁩

https://about.readthedocs.com/blog/2024/07/ai-crawlers-abuse/