deadlock Message Board
[ July Full List | Reply ]

Re: robots.txt
From Jim on 18 July '99
replying to robots.txt posted by Ari Manor

>In the monthly visitors report of my site,,
>most failures are related to requests for robots.txt

When search engine robots visit your site, they look for a robots.txt
file. If they don't find one, that's OK, it's optional. This file
is normally used to tell robots NOT to index certain pages on a site.

The short answer: don't worry about it.

Reply to this message

Required Details
Your name:

Optional Details
Home page title:
Home page URL:
(if you'd like to receive
automatic replies by email)

Your Message