Comment on Microsoft and Reddit Are Fighting About Why Bing’s Crawler Is Blocked on Reddit

<- View Parent
DaGeek247@fedia.io ⁨3⁩ ⁨months⁩ ago

My robots.txt has been respected by every bot that visited it in the past three months. I know this because i wrote a page that IP bans anything that visits it, and l also put it as a not allowed spot in the robots.txt file.

I've only gotten like, 20 visits in the past three months though, so, very small sample size.

source
Sort:hotnewtop