Jim's Message Board
[ Reply To This Message ]

The goods on Hotbot's spider
From Laurel Eby at Excel Translations on 17 June '98
I thought this might interest some of you. I wrote to Hotbot saying that several dead URLs from my company's site are listed in their index and asked how to get rid of them, and this is what they wrote:

"First, regarding the dead URLs. The add URL feature is meant for new URLs
that are not in the index, and resubmitting currently indexed URL can
actually delay the update process. When the URL shows up on the queue,
Hotbot will notice that it is already in the index and knock it off the
queue. the best thing to do is wait for the spider to crawl your site and
note that the specific pages are dead, and drop them."

So basically, if you're already in their index, resubmitting will not do any good.

Replies from other people:

Reply to this message:

Who Are You?
Your name:
Home page title:
Home page URL:

Your Reply: