Comment on Microsoft and Reddit Are Fighting About Why Bing’s Crawler Is Blocked on Reddit

<- View Parent
Moonrise2473@feddit.it ⁨4⁩ ⁨months⁩ ago

only if you don’t want any visits except from yourself, because this removes your site from any search engine

should write a “disallow: /juicy-content” and then block anything that tries to access that page (only bad bots would follow that path)

source
Sort:hotnewtop