It’d be more naive to have a robot.txt file on your webserver and be surprised when webcrawlers don’t stay away. 😂