Cover image for The Invisible Web : uncovering information sources search engines can't see
The Invisible Web : uncovering information sources search engines can't see
Sherman, Chris, 1958-
Personal Author:
Publication Information:
Medford, N.J. : CyberAge Books, [2001]

Physical Description:
xxix, 439 pages : illustrations ; 24 cm
Internet and the visible web -- Information seeking on the visible web -- Specialized and hybrid search tools -- invisible web -- Visible or invisible? -- Using the invisible web -- Case studies -- future : revealing the invisible web -- best of the invisible web -- Art and architecture -- Bibliographies and library catalogs -- Business and investing -- Computers and Internet -- Education -- Entertainment - Government information and data -- Health and medical information -- U.S. and world history -- Legal and criminal resources -- News and current events -- Searching for people -- Public records -- Real-time information -- Reference -- Science -- Social sciences -- Transportation.
Added Author:
Format :


Call Number
Material Type
Home Location
Item Holds
ZA4450 .S54 2001 Adult Non-Fiction Central Closed Stacks

On Order



Enormous expanses of the Internet are unreachable with standard web search engines. This book provides the key to finding these hidden resources by identifying how to uncover and use invisible web resources. Mapping the invisible Web, when and how to use it, assessing the validity of the information, and the future of Web searching are topics covered in detail. Only 16 percent of Net-based information can be located using a general search engine. The other 84 percent is what is referred to as the invisible Web--made up of information stored in databases. Unlike pages on the visible Web, information in databases is generally inaccessible to the software spiders and crawlers that compile search engine indexes. As Web technology improves, more and more information is being stored in databases that feed into dynamically generated Web pages. The tips provided in this resource will ensure that those databases are exposed and Net-based research will be conducted in the most thorough and effective manner.

Author Notes

Gary Price is a reference librarian at George Washington University. He lives in Vienna, Virginia. Chris Sherman is the director of the guide to Web searching on and president of Searchwise, a consulting firm. He is the author of the CD-ROM Handbook and a frequent contributor to Online magazine. He lives in Los Angeles. Danny Sullivan works for

Reviews 1

Choice Review

Today, the Web is the first place people look for information. Unhappily, the retrieval tools used--search engines--are losing their effectiveness. An increasing percentage of the Web is "invisible" to search engines because of the technical and logistic limitations of current search engine technology. Unless users know where and how to search, much of the Web's most authoritative content is effectively invisible because it is so difficult to locate. Chris Sherman (associate editor of ) and Gary Price (compiler of Price's List of Lists and The Virtual Acquisition Shelf and News Desk ) present an overview of what is known about search engine technology by focusing on problems in applying this technology to the rapidly expanding, evolving Web. Dynamic database-driven Web sites, Flash and Shockwave technology-driven Web sites, and password-protected Web pages (with free registration) are growing problems for today's search engines. Twenty-five "invisible" Web categories are explored, including authoritative dictionaries, historical documents, and philanthropy and company directories. The balance of the work consists of an annotated directory of almost 1,000 Web sources arranged by broad subject categories (e.g., art and architecture, business and investing, education, entertainment, government information, health information, legal information, real-time information, science) and a brief glossary of commonly misunderstood information retrieval terms. Information is kept current at . Strongly recommended for librarians and reference service staff in public, academic, and corporate libraries. J. A. Buczynski University of Miami

Table of Contents

Figures and Tablesp. xiii
Forewordp. xv
Acknowledgmentsp. xix
Introductionp. xxi
The Invisible Web in a Nutshellp. xxii
What to Expect from This Bookp. xxiii
Boldly Go Where No Search Engine Has Gone Beforep. xxvi
About www.invisible-web.netp. xxix
Chapter 1 The Internet and the Visible Webp. 1
How the Internet Came to Bep. 2
Early Net Search Toolsp. 3
Enquire Within Upon Everythingp. 8
Weaving the Webp. 10
Early Web Navigationp. 12
The First Search Enginesp. 13
Chapter 2 Information Seeking on the Visible Webp. 17
Browsing vs. Searchingp. 19
Web Directoriesp. 22
How Web Directories Workp. 22
Issues with Web Directoriesp. 24
Search Enginesp. 26
How Search Engines Workp. 26
Issues with Search Enginesp. 32
Search Engines vs. Directoriesp. 36
Chapter 3 Specialized and Hybrid Search Toolsp. 37
Targeted Directories and Focused Crawlersp. 38
Targeted Directoriesp. 39
Focused Crawlersp. 41
Vertical Portals (Vortals)p. 43
How to Find Vortalsp. 44
Metasearch Enginesp. 44
Issues with Metasearch Enginesp. 45
Value-Added Search Servicesp. 46
Alternative Search Toolsp. 48
Browser Agentsp. 48
Client-Based Search Toolsp. 50
Web Ringsp. 51
Fee-Based Web-Accessible Servicesp. 52
Next Stop: The Invisible Webp. 52
Chapter 4 The Invisible Webp. 55
Invisible Web Definedp. 56
Why Search Engines Can't See the Invisible Webp. 62
Four Types of Invisibilityp. 70
The Opaque Webp. 70
The Private Webp. 73
The Proprietary Webp. 73
The Truly Invisible Webp. 74
Chapter 5 Visible or Invisible?p. 77
Navigation vs. Content Sitesp. 78
Direct vs. Indirect URLsp. 79
The URL Testp. 80
Specialized vs. Invisiblep. 82
Visible vs. Invisiblep. 83
The Library of Congress Web Site: Both Visible and Invisiblep. 86
The Robots Exclusion Protocolp. 89
Chapter 6 Using the Invisible Webp. 91
Why Use the Invisible Web?p. 92
When to Use the Invisible Webp. 95
Top 25 Invisible Web Categoriesp. 96
What's NOT on the Web--Visible or Invisiblep. 103
Spider Traps, Damned Lies, and Other Chicaneryp. 105
Keeping Current with the Invisible Webp. 109
Build Your Own Toolkitp. 111
Chapter 7 Case Studiesp. 115
Case 1 Historical Stock Quotesp. 115
Case 2 Patent Informationp. 117
Case 3 Real-Time Trackingp. 119
Case 4 Locating an Out of Print Bookp. 120
Case 5 Telephone Numbers and Zip Codesp. 121
Case 6 Finding Online Imagesp. 122
Case 7 Investment Researchp. 123
Case 8 The Invisible Web Fails to Deliver!p. 124
Chapter 8 The Future: Revealing the Invisible Webp. 127
Smarter Crawlersp. 128
The Promise and Pitfalls of Metadatap. 129
Beyond Textp. 130
Delving into Databasesp. 130
Hypertext Query Languagesp. 132
Real-Time Crawlingp. 132
Long Live the Invisible Webp. 133
Chapter 9 The Best of the Invisible Webp. 135
Invisible Web Pathfindersp. 135
An Invisible Web Directoryp. 137
Frequently Asked Questions about the Directoryp. 138
In Summary: The Top 10 Concepts to Understand about the Invisible Webp. 142
Chapter 10 Art and Architecturep. 145
Architecturep. 146
Artistsp. 147
Galleries on the Webp. 148
Gateways to Art and Architecture Resourcesp. 151
Referencep. 152
Chapter 11 Bibliographies and Library Catalogsp. 153
Bibliographiesp. 154
Library Catalogsp. 160
Chapter 12 Business and Investingp. 163
Company Information and Researchp. 164
Consumer Resourcesp. 169
Economics--United Statesp. 169
Economics--Worldp. 173
Financial Institutionsp. 175
General Business Resourcesp. 176
Government Contractsp. 177
Industry-Specific Resourcesp. 178
Investment Resourcesp. 182
Jobs and Career Informationp. 185
Lookup Servicesp. 187
Marketing Resourcesp. 188
Pension Resourcesp. 189
Personal Financesp. 190
Philanthropy and Non-Profit Resourcesp. 190
Research and Developmentp. 192
Real Estatep. 193
Tariffs and Tradep. 194
Trade Shows and Conventionsp. 196
Chapter 13 Computers and Internetp. 199
Computers and Computingp. 200
Internet Resourcesp. 203
Chapter 14 Educationp. 207
Classroom and Teacher Supportp. 208
Directories and Locatorsp. 209
Financial Information and Scholarshipsp. 212
General Education Resourcesp. 213
Statisticsp. 215
Chapter 15 Entertainmentp. 217
Amusementsp. 218
General Entertainment Resourcesp. 218
Movies and Cinemap. 219
Musicp. 221
Performances and Eventsp. 224
Chapter 16 Government Information and Datap. 227
Directories and Locatorsp. 228
General Government Resourcesp. 230
Government Documentsp. 231
Government Officialsp. 234
Government Programsp. 235
Politics, Policy, and International Relationsp. 236
Statisticsp. 238
Chapter 17 Health and Medical Informationp. 241
Diseases and Conditionsp. 242
Imagesp. 246
Healthcare and Medical Informationp. 247
Healthcare Professional Resourcesp. 249
Locatorsp. 251
Nutritionp. 253
Patient Information and Consumer Resourcesp. 254
Pharmaceutical Drugsp. 256
Researchp. 257
Workplace Health and Safetyp. 259
Chapter 18 U.S. and World Historyp. 261
United States Historyp. 262
World Historyp. 267
Chapter 19 Legal and Criminal Resourcesp. 271
Attorney and Law Firm Locatorsp. 272
Crime and Criminalsp. 273
Decisionsp. 273
Documents and Recordsp. 274
General Legal Resourcesp. 274
Intellectual Propertyp. 276
Laws, Codes, and Treatiesp. 278
Chapter 20 News and Current Eventsp. 283
Audiop. 284
Directoriesp. 284
News Search Resourcesp. 285
Videop. 288
Chapter 21 Searching for Peoplep. 291
Famous and Historical Peoplep. 292
Genealogy Resourcesp. 293
Group and Affiliation Directoriesp. 294
Online White Pages and Lookup Toolsp. 297
Veterans and Currently Serving Militaryp. 299
Chapter 22 Public Recordsp. 301
General Public Records Resourcesp. 302
Location-Specific Public Recordsp. 303
Chapter 23 Real-Time Informationp. 311
Environmentp. 312
Governmentp. 313
Miscellaneous Trackingp. 314
Space and Satellitep. 314
Stock Quotesp. 315
Transportationp. 315
Weatherp. 317
Chapter 24 Referencep. 319
Associationsp. 320
Awardsp. 321
Booksp. 322
Calculatorsp. 323
Consumer Resourcesp. 324
Dictionaries, Glossaries, and Translation Resourcesp. 325
Food and Beveragesp. 327
General Reference Resourcesp. 328
Journals and Periodicalsp. 331
Library/Online Searchingp. 332
Locatorsp. 333
Maps and Geographyp. 335
Sportsp. 338
Travelp. 339
Weatherp. 341
Chapter 25 Sciencep. 343
Agriculturep. 344
Biologyp. 345
Botanyp. 347
Chemistryp. 348
Earth Sciencesp. 350
Energyp. 353
Engineeringp. 355
Environmentp. 356
General Science Resourcesp. 361
Mathematics and Physicsp. 362
Oceanographyp. 362
Research and Developmentp. 363
Space and Astronomyp. 365
Weather and Meteorologyp. 367
Chapter 26 Social Sciencesp. 369
Anthropologyp. 370
Archaeologyp. 370
Demographicsp. 371
Development Resourcesp. 373
General Resourcesp. 374
Gender Studies and Datap. 375
Latin Americap. 376
Military Resourcesp. 377
Psychologyp. 377
Research and Developmentp. 378
Religionp. 378
Chapter 27 Transportationp. 381
Airp. 382
Automobilep. 384
General Transportation Resourcesp. 384
Maritimep. 386
Railroadp. 387
Glossaryp. 389
Referencesp. 397
About the Authorsp. 401
Indexp. 403