deadlock Message Board
[ March Full List | Reply ]

robots.txt
From inachu on 20 March '01
I noticed palm(http://www.palm.com/robots.txt) has their robots.txt file just uses an asterisk to ban spiders.

Does the asterisk work or must you specify every spider agent?




Inachu




Reply to this message

Required Details
Your name:
Subject:

Optional Details
Home page title:
Home page URL:
Email:
(if you'd like to receive automatic replies by email)

Your Message