This type of engine uses a robot or a spider (there's an arcane difference
between the two that isn't very interesting) which constantly
roams the World Wide Web searching for new or updated pages (what a
wonderful job eh?) When the robot visits a page it reads the whole thing,
including any sub-pages on the same server connected to the page it's
reading, then it visits all the external links on the pages it's
read...and so on, you get the idea.
A robot can only follow links. It finds pages in two ways:
by visiting pages it's been told about (ie. on its submission form) or by
following links from other pages. So, if your page isn't linked from
anywhere else and you haven't registered it, there's no way it'll ever
be found — it'll be invisible. You can actually ensure that most of the
deep engines don't index your pages by using special "no follow" META tags
if you want to. Tagmaster will do this for
The point is that all the while it's storing every word on every page it
visits, then compressing the information down and storing it all in its
database. Now, you may be thinking "Wow! I'll always use a deep search
engine to find things in future." but in practice the deep ones are useful
for finding specific or obscure information using long search strings, but if
you're looking for something popular like games then you'll probably get a
search results list running into the millions which isn't much use.
What this means to you:
Don't make the mistake of thinking that the robots will get to your site
eventually. Register it. Normally, all you'll need to do is fill in your
URL and nothing else.
Easy. To attract the attentions of deep search engines (to put your page
high on results lists) you need to tailor your page itself which is discussed
This is the complete list of deep search engines that use robots to index pages:
You see, there aren't many of them.