I am developing this page as a subject gateway of online research resources related to Web-based Information Retrieval and Information Seeking on the Web in general and on the Invisible Web in particular. If you know any other resources which would be related to these topics please let me know .
An Annotated List of Resources on the Invisible Web:
1. Sherman, C. & Price, G. (2001). The Invisible Web: Uncovering Information Sources Search Engines Can't See. CyberAge Book.
This is the only comprehensive published book entirely about the Invisible Web, which I have seen so far. While a huge amount of the Web-based information is hidden under the shade of the Invisible Web, the issue of invisibility has remained still invisible in the majority of related literature. As the title of the book indicates clearly, its main theme is about invisible web. The book is divided into two major parts including: Revealing the Invisible Web in eight chapters and The Invisible Web Directory in nineteen chapters.
In the beginning chapters, it provides the reader with concise but very useful information about the World Wide Web in general and various kinds of search tools in particular. Information seeking on the visible Web and specialised search tools are two profound chapters of the book introducing the mechanism and associated issues of general-purpose search engines and some other existing search facilities on the Web. Then the book explains what is the concept of invisibility on the Web and why it exists. It follows by illustrating four different kinds of invisibility. It also takes a look to the future of this area by mentioning some topics such as smarter crawlers and delving into databases. The second part of the book provides the reader with a directory of Invisible Web categorised by subject. Invisible-web.net is a website which acts as a supplement of this book and in fact provide the reader with a web-based version of the second part of this book which is a subject directory of the Invisible Web resources.
This is another book which is entirely dedicated to the Invisible Web. This book is a concise and useful introduction concerning the issue of the invisibility on the Web. The book introduces the invisible Web phenomenon and concisely clarifies the main reasons causing invisibility on the Web. Then it explains what this part of the Web contains and why it is important for the Web searchers to be aware of the presence of the invisible Web and how they can search through this section of the Web environment. The book is a good source for searchers and can make them familiar with this issue and guide them how they can increase their information accessibility on the Web and introduces a number of directories and search tools which are useful for locating information through the opaque side of the Web. Paul Pedley develops a Weblog entitled KeepingLegal site and newsletter which appears on Law librarian news and the related issues. He also has experiences of teaching about the Invisible Web in a number of workshops and training courses.
In spite of the importance of the hidden sides of the Web, there are quite a few useful resources on the Web containing the helpful instructions to make Internet users aware about the presence of the Invisible Web and guiding them to a number of practical ways to penetrate into this resources. Those Dark Hiding Places: The Invisible Web Revealed, is a useful source of information with this approach and has been created by Robert J. Lackie, who is an assistant professor-librarian, at Rider University.
4. The Deep Web
In the Internet tutorial of the Library of University at Albany state University of New York, Laura Cohen has explained about The Invisible Web term and other related phrases including the Deep Web. It is a good Web page to explain differences between these terms and illuminate some aspects of this topic.
The Deep Web: Surfacing Hidden Value is one of the most cited resources about the quality and quantity of the Deep Web which is the biggest part of the Invisible web. This a white paper which attempts to quantify the size and characteristics of the hidden side of the Web. It has been published in 2001 and regarding the growth rate of the electronic data on the Web the presented statistics and figures seem out of date. However, this is the only comprehensive web-based documents which provides us with some figures about the estimated quantity of the Deep Web.
Although the resources hidden in the invisible web is a challenge for web users, it can be assumed as an opportunity for librarians and information professionals to carry out their responsibilities in new era of electronic resources. Given to this fact that the majority of end users usually use search engines as their primary search tools on the Web, the invisible web resources will remain invisible for them. So, how librarians can help them in their information seeking process. This is a broad area of discussion about what is the real role of librarians in helping people to penetrate into the opaque territory of the Web and locating their information needs more efficiently. Brian Smith in his useful paper “Getting to Know the Invisible Web” has discussed about the invisible web in general and this issue as well.
Locating information on the Web is usually a double-edged procedure. It can be an easy and fast process or a difficult and frustrating experience. Nevertheless, everybody can enhance his/her search process through learning a few simple tactics. In fact, there is a difference between information seeking and information mining on the Web. We need to learn how to delve into web-based resources properly, or in a better term, in a more efficient manner. Mining Deeper Into the Invisible Web is an article written by Diana Botluk including some guidelines for the end users to locate their information needs in a more efficient way. In Addition she has published another article entitled Exposing the Invisible Web which is useful as well.
There is a strong relationship between search engines’ function and the invisible Web. In fact in the most widely accepted definition, the Invisible Web refers to the information resources available on the Web environment which cannot be retrieved through the general-purpose search engines. In fact the information resources on the Invisible Web are only invisible for those Internet searchers who limit their search tools to general purpose search engines like Google, Altavista or other ones. Beyond Google: The Invisible Web presented by Jane Devine and Francine Egger-Sider is a good introduction to this issue. Additionally, there are interesting examples, some fast facts, a glossary and a comprehensive enough Invisible Web-ography on this site.
Danny Sullivan is the chief editor of Search Engine Report which is an online magazine and is being published monthly about almost everything on the search engines including the features, facilities and limitations of the these search tools. He has published many papers about search facilities on the Web and this paper (Invisible Web Gets Deeper) is about the Invisible Web growing rate and its importance. In this paper Sullivan attempts to clarify and put emphasis on this reality that we cannot simply ignore the invisible Web resources, because in terms of size and quality these information sources are very important. At the end of this paper there are some useful links to the related websites.
Deep Web Research is a Weblog developed and maintaining by Marcus P. Zillman from Virtual Private Library, This Weblog includes many links to other Web-based resources on the Invisible Web topic in general and the Deep Web in particular.
The phrase of the Invisible Web might seem ambiguous or even misleading for people. Probably it sounds that the resources in this part of the Web are unseen or and literally invisible. However, these resources can be retrieved when people know where they should look for them. This is one of the area of work for librarians and information experts to help people and teach them to locate their information needs though the invisible Web. Seeing the Invisible Web provides web searchers with a brief explanation about the Invisible Web and guide them to search the Web more effectively.
Searching the Invisible Web is another example that librarians can assist people to find their way through the dark sides of the Web. Although this is just a brief description of the Invisible Web and includes a few links to some useful resources, it can be very helpful for those Web searchers who are not familiar with this phenomenon and limit their search to search engine facilities.
K. Wiseman has created this webpage to provide the readers with basic definitions of the Invisible Web and making some links to a number of relevant resources. This is a good page to learn an introduction on this topic.
This is a meta-search engine powered by Intelliseek company which not only is able to search a number of search engines including Altavista, Alltheweb, Lycos, Metacrawler and so on simultaneously but also possess some interesting features and search facilities providing searchers with more sophisticated search facilities. There is also a targeted subject directory which is useful to browse the Web environment based on different subject.
This is another successful example of the librarian roles in helping people to locate their information needs through the Invisible Web and making them aware of the existence of this phenomenon and the factors causing it. This tutorial page has been developed by UC Berkeley Teaching Library Internet Workshops and provide readers with a lot of useful information about the Invisible Web. This is a good source for those people who would like to know an introduction about the meaning of the invisible Web and learn how to cope with this barrier in their information seeking procedure on the Web.
This page is similar to a FAQ (Frequently Asked Question) source about the invisible web and include a number of links to related resources.
This is a search engine which has a directory as well. In spite of its fascinating title, the search results are not relevant. This search engine is not recommended here and I just mention it as a source which has taking the invisible web into account.
Gary Price who is one of the authors of The Invisible Web: Uncovering Information Sources Search Engines Can't See published by CyberAge Book in 2001 is developing this site. This websites is similar a newsletter about all topics which are related to web searching and the Invisible Web. There is a reminder mailing list over there also that anybody who like to remain up-to-date about news on the Invisible Web can subscribe there.
This is a brief article written by David Coursey Executive Editor of AnchorDesk. He attempts to attract the readers attention to the issue of the Invisible Web and has recommended them to carry out their search sessions through new search facilities which are capable to penetrate into the depth of the Web.
Deepweb.com contains a brief description about the meaning of the Deep Web and contains a number of useful web-based resources on this issue. A few search facilities have been introduced and recommended in this site which are able to penetrate into deeper parts of the web and reduce the level of invisibility for people.
This is a Weblog which is entirely on the Invisible Web topic and is being developed by me since August 2003. There are many links to other resources in this issue.
This list is under construction now and new resources will be added hopefully in near future.