|
Get Streetwise
|
2. How They Work
If you have the right software there are several hundred more at your disposal.
Deep Search Engines- This type of engine uses a robot or a spider (there's an arcane difference between the two that isn't very interesting) which constantly roams the World Wide Web searching for new or updated pages (what a wonderful job eh?) When the robot visits a page it reads the whole thing, including any sub-pages on the same server connected to the page it's reading, then it visits all the external links on the pages it's read...and so on, you get the idea.
A robot can only follow links. It finds pages in two ways: by visiting pages it's been told about (ie. on its submission form) or by following links from other pages. So, if your page isn't linked from anywhere else and you haven't registered it, there's no way it'll ever be found — it'll be invisible. You can actually ensure that most of the deep engines don't index your pages by using special "no follow" META tags if you want to. Tagmaster will do this for you.
- The point is that all the while it's storing every word on every page it visits, then compressing the information down and storing it all in its database. Now, you may be thinking "Wow! I'll always use a deep search engine to find things in future." but in practice the deep ones are useful for finding specific or obscure information using long search strings, but if you're looking for something popular like games then you'll probably get a search results list running into the millions which isn't much use.
What this means to you:
Don't make the mistake of thinking that the robots will get to your site eventually. Register it. Normally, all you'll need to do is fill in your URL and nothing else. Easy. To attract the attentions of deep search engines (to put your page high on results lists) you need to tailor your page itself which is discussed later.
This is the complete list of deep search engines that use robots to index pages:
- You see, there aren't many of them.
Standard Search Engines (Directories)- Don't be put off by the heading, these are often more useful than the deep engines for finding useful, relevant information on general or popular topics.
The only information held in the database is what is input by the people, like yourself, registering their pages by the use of input forms provided by the administrators of the engine. This makes the standard search engines the focus of your attention when it comes to submitting your pages.
Examples of standard search engines:
- EINet Galaxy
- New Rider's Yellow Pages
- Starting Point
Categorised Listings- These sites are arranged much the same as the Standard Directories, except no search engine is involved. Information is held in 'categories' just like the directories on your hard disk, and the categories contain lists of titles, linked to the URL, with short descriptions underneath. The most important thing here is to place your page in the most relevant category where you think people will look for your product.
There are thousands of examples of these listings, often called "free-for-all links pages". They don't get a huge amount of traffic, but if you submit to enough of them the cumulative traffic will add up. You definitely need software to take advantage of these, it's too much to do by hand.
Yahoo!- This is the undisputed 'king of the search engines' and deserves a section to itself. Yahoo! started off as a categorised listing (it still is) which evolved by having a search engine added once it became mega-popular. Around mid 1996, they started getting worried about the popularity of the up-and-coming Alta Vista, so they pulled yet another master stroke by joining forces with them to form an alliance which was virtually invincible (in late 1998 they switched to a similar partnership with Hotbot). If a search doesn't find a listing within Yahoo! then it defaults to the larger, robot-generated database transparently.
-
Note: people often make the mistake of thinking they have a Yahoo! listing
when they see their page(s) appearing in the results list, when in fact they're
looking at the larger database, which isn't the same as a true Yahoo! listing.
- This type of engine uses a robot or a spider (there's an arcane difference between the two that isn't very interesting) which constantly roams the World Wide Web searching for new or updated pages (what a wonderful job eh?) When the robot visits a page it reads the whole thing, including any sub-pages on the same server connected to the page it's reading, then it visits all the external links on the pages it's read...and so on, you get the idea.
| Free newsletter: subscribe |  |