Re: robots.txtFrom Jim on 18 July '99
replying to robots.txt posted by Ari Manor
>In the monthly visitors report of my site, http://www.zooz.co.il,
>most failures are related to requests for robots.txt
When search engine robots visit your site, they look for a robots.txt
file. If they don't find one, that's OK, it's optional. This file
is normally used to tell robots NOT to index certain pages on a site.