---

[ July Full List | Reply ]

---

robots.txt From Ari Manor at ZOOZ on 17 July '99 Hi there! In the monthly visitors report of my site, http://www.zooz.co.il, most failures are related to requests for robots.txt Does this mean I have to create some special robots.txt for spiders? Which text should be written in this robots.txt file? A quote from the failure report is below. Thanks for your help, Ari ========== Failure Report Listing files, sorted by the number of failed requests. #reqs: file -----: ---- 18: /robots.txt 1: /heb/intro.html 1: /eng/intro.html 1: /intro.htm 1: /

Re: robots.txt - Jim - 18/07/99 Re: robots.txt - Aøï - 19/07/99

Reply to this message
Required Details
Your name:
Subject:

Optional Details
Home page title:
Home page URL:
Email: (if you'd like to receive automatic replies by email)
Your Message
>Hi there! > >In the monthly visitors report of my site, http://www.zooz.co.il, >most failures are related to requests for robots.txt >Does this mean I have to create some special robots.txt for spiders? >Which text should be written in this robots.txt file? > >A quote from the failure report is below. > >Thanks for your help, >Ari >========== > >Failure Report > > >Listing files, sorted by the number of failed requests. > > >#reqs: file >-----: ---- > 18: /robots.txt > 1: /heb/intro.html > 1: /eng/intro.html > 1: /intro.htm > 1: /