deadlock Message Board
[ April Full List | Reply ]

Lycos Robot and dynamic pages
From Baz on 19 April '99
Just been checking my logs, and I noticed something strange.

I have an ASP page which is only ever called with a querystring
ie. newsitem.asp?item=xyz

Nothing has ever indexed this page, I guess because the search engine spiders detected the ? and
ignored it. BUT, the lycos spider has been poking it's nose in and has been requesting the page.

Does this now mean that Lycos is indexing dynamically created pages? Anyone else notice if the other
search engines have started doing the same?


Reply to this message

Required Details
Your name:

Optional Details
Home page title:
Home page URL:
(if you'd like to receive
automatic replies by email)

Your Message