Searching the Web

Knowing which search engine does what can be a huge time-saver. So which search engine does what? In brief, here's a quick run-down of some well-known general topic search engines. More comprehensive details are available from The Spider's Apprentice.

Comparison of Search Engines:

Note: Some of the information on this page was taken from The Spider's Apprentice by Linda Barlow (lindabarlow@monash.com).

Google is one of today's most popular search engines. It is a keyword search engine and, in addition to the terms you enter, will also return close variations of your terms, such as "dietary" if you entered only "diet." By default, it looks for all of the terms that you include and is not case sensitive. It is also easy to use clarifiers such as AND, NOT, and "<exact phrase>" to refine your search.

Yahoo is a actually a directory--a subject index. It's an attempt to catalogue and organize the entire Web. Search on a subject or topic. If Yahoo can't give you a hit, it automatically sends your query to its partner Alta Vista for a search through that index (and can also send your query to most of the other search engines listed here).

Excite is a concept-based search engine. This means it will try to figure out what you mean...instead of just searching on exactly what you say. This gives Excite a great deal of flexibility. Like Infoseek (see below), Excite now includes the option of a "Personal Page," where you can define all the subjects that you are most interesting in searching.

Alta Vista is a powerful keyword search engine. It allows you to search on phrases of more than one word if you put the phrase into quotations marks, i.e., "health care facilities" or "Bill Gates." Alta Vista has lots of search-refining options. It's big and fast, and very popular among dedicated web-surfers.

Lycos is big and slow, but solid, thorough and comprehensive. Its main flaw is a limited ability to refine a search. Lycos is gradually becoming more like Yahoo, acting as a Web Directory as well as a search engine.

Webcrawler is fast and cool. If you don't object to link-hopping to check out your hits, Webcrawler returns useful and relevant hits on simple queries. Its latest developments include more sophisticated query-refining techniques.

MetaCrawler is a metasearch engine, which means that when it performs a search, it looks for your search terms in multiple search engines, such as Google, Yahoo!, Ask Jeeves, About, Teoma, FindWhat, LookSmart, and others. Thus, the results you see when using MetaCrawler are what it sees as the best results from the many engines it searches.

HotBot uses the former Inktomi technology and so far, it looks pretty cool, offering different types of searches and excellent search refining.