robots.txtis a bit like asking bots to not visit my site; with.htaccess, you’re not asking
Blocking Bots with Nginx
Submitted 1 year ago by poVoq@slrpnk.net to selfhosting@slrpnk.net
https://rknight.me/blog/blocking-bots-with-nginx/
Comments
Zachariah@lemmy.world 1 year ago
nek0d3r@lemmy.world 1 year ago
My robots.txt has definitely only partially helped. I’ll have to see if I can do the same with my Caddy setup.