Web Searching

Laura E. Ray, MA, MLS
Educational Programing Librarian
Cleveland-Marshall College of Law

September 2004

When using research methodologies, the WorldWideWeb may be viewed as one electronic information medium to be considered with print, audiovisual, and other electronic information media.  In addition, as in other information media, it can be difficult to find the information you want.  This "Web Searching" guide will help you to be a better Web researcher, as well as help you to better evaluate Web sites you find in your research. 

Web Site Evaluation

A WorldWideWeb "site" is composed of many "pages."  Current size estimates of the Web put it at 3 billion pages on 20 million sites.  [For statistics, news, and research information on the Web, consult ClickZ]

Maybe you found a Web site that seems to have information you want.  How can you know if the Web site is reliable?  You can probably trust sites from established organizations, but what about Web sites of organizations you know nothing about?
 

Web Site Home Page and Site Index

When evaluating a Web site, go to its "home page," or opening page of the site, and look for these key pieces of information:

• Purpose
• Scope of Services and Information
• Navigation Methods
• Help Information

Beyond the home page and site index, consider the following criteria when evaluating a Web site:

• Credibility / History - who created the site; are they a reliable authority; who are their affiliations, partners, or sponsors?
• Purpose / Philosophy - why was the site created?
• Audience / Relevance - who was the site designed for?
• Content - what are the site's topics or subjects?
• Scope / Context / Coverage - how much information on a topic does the site cover; does the site report its size and growth rate?
• Selection Criteria / Critical Thinking / Objectivity / Censorship - how is information selected for the site?
• Accuracy / Documentation - does the site document its sources?
• Currency / Updating - how often is the site's information updated and what is the scope of that updating?
• Writing Quality - can you cognitively understand the site's information?
• Design / Presentation Format - can you visually or auditorially understand the site's information?
• Stability / Continuity / Maintenance - can you consistently connect to the site?
• Accessibility - is the site accessible via different browsers; does the site have "text only" or "no frames" versions?
• Interface - can you understand the site's graphical and textual controls; how quickly do these controls operate?
• Navigation / Site Index / Searchability - can you understand how to find information on the site?
• Connectivity - does the site link to other helpful sites?
• Help Information / Frequently Asked Questions / Customer Support
• Usefulness / Value-to-Cost Ratio - how does the site compare to other analogous sites; if it's a fee-based site, is its information worth the cost?

Web Site Reviews

Internet Scout Project
    Produces "The Scout Report," which announces and reviews Web sites and mailing lists.
    "Table of Contents" for current Report.
    "Archives" contain approximately 17,000 reports.
             At "Archives" page, also note ability to browse for reviews by Library of Congress Subject Headings.
                    [eg, "Law - United States - cases"]
    "Archives" are searchable; "Advanced Search" also available.
 

Web Searching - General Issues

The Web offers a variety of good search services.  Many provide "value-added" services, such as customized display of search results.  However, it is important to remember that economic forces are quite active on the Web, and search services do not reach all Web-based information.  [Web search services index an estimated 20-50% of Web.  See Invisible Web as well as page 53 of  The Invisible Web: Uncovering Information Sources Search Engines Can't See available from the Cleveland-Marshall College of Law Library.]
 

Web Search Service Trends

Over the last decade, several trends have emerged in Web search service operations:

• Partnerships - partnering with organizations, such as "amazon.com," to offer easy material purchase and other customer services.
• Advertisements - using pop-up ads to cover maintenance expenses or make additional income.
• Fee Services - offering fee-based companion services such as a full-text database.
• Charges for preferential listings - selling high positions in search results lists.  [Responding to complaint filed by Commercial Alert, the US Federal Trade Commission sent letter, on 27 June 2002, to leading search engine companies warning them to adequately disclose and distinguish paid placements from unpaid ones.  See  Commercial Alert Complaint Letter]
• Customized search form and results display - offering capability of personalizing how you search and see search results; "cookies" (ie, information stored on your computer by a Web site) make this possible.

Web Search Service Problems

Despite their wonderful capability of finding information, there can be problems with the information from Web search services.  Web Search Directories cover less of the Web than Engines, but tend to have fewer problems because human beings compile verified information about Web sites.  Web Search Engines cover more of the Web than Directories, but they electronically compile information that may or may not be verified.  All Web search services can be subject to the following problems:

• Inadequate updating and relocating leads to retrieval of many invalid and duplicate URLs.  [The URL - uniform resource locater - is the "address" of a Web site or page.]
• Because each search service covers different parts of the Web, as well as indexes overlapping parts in different ways, searchers need to use multiple search services to be really thorough.
• "Popular," rather than the best, Web sites tend to get indexed more.  ["Popular" site is one that other sites link to; the more sites that link to a site, the more "popular" that site is.]
• It can take months for new site to get indexed.
• Information in frames or image maps is often not indexed.
• <alt> tags describing graphics are indexed, but not the words in the graphics themselves.
• Inaccessible information in "Invisible Web" (eg, password sites).  [More on this in  Invisible Web]
• Partnership constraints may affect information provided.
• Annoying, distracting advertisements.
• Slow, congested traffic! [WorldWideWait]

Web Search Service Evaluation

Search Engine Showdown
    Search service review Web site produced by Greg R. Notess.
    Provides reviews, analyses, and comparative information.

Search Engine Watch
    Search service review Web site created by Danny Sullivan, who continues to edit the site for Jupitermedia Corporation.
    Provides statistics, reviews, and comparative testing.
    Includes free daily "SearchDay" and monthly "Search Engine Report" newsletters.

In addition to consulting the above review sites, consider the following criteria when deciding whether a Web search service meets your needs:

• Selection criteria or human involvement in indexing, reviewing, and screening information.
• "Refresh" frequency - how often is the directory information updated; how often does the engine robot "recrawl" to refresh information?
• Interface - can you understand the search service's graphical and textual controls; how quickly do these controls operate; is there an advanced search capability?
• Search Tips / Help Information.
• Boolean Logic Connectors (eg, "and," "or," "not") - can you use these to construct a search statement?
• Proximity Searching (eg, "near," "adj") - this connector is especially desired for full-text searching.
• Phrase Searching.
• Case Sensitivity - does the service understand upper and lower case?
• Punctuation Sensitivity - does the service understand punctuation?
• Truncation Capability - can the service retrieve varying forms of word (eg, where "child*" retrieves "child" "children" etc.)?
• Field Searching - can you limit a search to a particular portion of Web site?
• "Stop Words" (eg, to, be) - does the service not search certain common words?
• Results Display - does the service limit search results in any way; how does the service rank search results?
• Cost / Fee.

Web Searching Principles and Guidelines
 

Key Principles

• If first thirty hits not on point, change your search statement and/or use different search service.
• Evaluate retrieved sites based on criteria outlined in  Web Site Evaluation
• Bookmark appropriate and valuable Web sites.

General Guidelines

• Write down what you are seeking - combine keywords into search statement - before going online.
• Consider browsing a Search Directory before using a Search Engine.
• Check search service's tips and help information.
• Keep search simple - much irrelevant information often retrieved with complex searches, because the searcher could not effectively combine keywords.
• Use several synonyms of keywords.
• Start specific; only use general terms if necessary.

  Narrow search - will have fewer items retrieved, but they will likely have high relevance.
  Broad search - will have many items retrieved, but most will likely have low relevance.

• Enter most important concept first.
• Use phrases - phrases are usually enclosed in quotation marks.
• Boolean Connectors -

  Use upper case letters.

  Check help information for default connector (often defaults to "and").

  Often use + in front of word for "and" (ie, includes term).

  Often use - in front of word for "not" (ie, excludes term).

• Proximity Connector (eg, "near," "adj") - this connector especially desired for full-text searching.
• Truncation symbol usually an asterisk (*).
• Use parentheses to combine keywords, similarly to data combinations in algebra or deductive logic.
• If available, use "field" searching - Web site title and URL particularly helpful.
• If available, use "limit/refine" capability.
   
   

Web Search Directories

Web Search Directories are created by human beings who identify Web sites and list them in a subject classification.
You can browse or search a Web Search Directory.
Usually the main page of a Web sites, rather than multiple pages of that site, is listed in search results.
Annotations and evaluations of Web sites are often included in search results.

Open Directory Project
    Developed and maintained by 30,000 volunteer editors.
    Netscape owns the copyright to ODP's compilations, but freely grants license to them.
    Thus, ODP is used by many search engines as their subject directories.

    Check "Law" sub-category under "Society" category.

Yahoo!
    Founded by David Filo and Jerry Yang in 1994; now a corporation based in Sunnyvale, CA.
    Charges "commercial" site annual fee to be listed.
    Recently acquired search engines "All the Web" and "AltaVista" and started "Yahoo!Search."  [See Web Search Engines ]

    Check "Government" category.

WWW Virtual Library
    First search directory founded by Tim Berners-Lee, creator of HTML and the Web.
    Maintained by volunteers.

    Check "Law" category.
 
 

Web Search Engines

To establish its information bank, a Search Engine's search software sends out "robots" (or "spiders" or "crawlers") that use HTTP to request data from Gopher, FTP, and HTTP servers.  [HTTP - HyperText Transmission Protocol; FTP - File Transfer Protocol.]  Data is indexed and stored; main page and additional pages of site are indexed.

Note:  When you use a Search Engine, you are only searching information indexed and stored by that Engine.

Google and Yahoo!Search are two leading search engines.  Both have the following features:

• Regular and "advanced" searching
• Searches Web pages as well as image, PDF, and other file types (eg, Rich Text Format, PowerPoint).
• When using Boolean operators, defaults to "and"; also supports Boolean operators  "or" and "not"
• "+" for "must include" and "-" for "exclude"
• Phrase searching
• Not case sensitive
• Field searching
• File format (eg, pdf) searching
• Supports 3-month, 6-month, and year date restrictions
• Can search in at least 35 languages
• Translates text or Web page from at least English to French, German, Italian, Portuguese, and Spanish; these five languages into English; as well as German to French and French to German
• "Mature" or "adult" filter option

"ohio legal research"
        About 130 Web pages will be retrieved.
"legal research" +ohio
        About 135,000 Web pages will be retrieved.
"legal research" AND ohio
        About 134,000 Web pages will be retrieved.

Yahoo!Search
    Also searches Yahoo! directory, as well as other Yahoo! portal databases (eg, Yahoo!News).
    Supports search nesting within parentheses.
    Can use stop word (eg, "a") as a "wildcard" within a phrase search.
    Also translates text or Web page from English to Chinese, Dutch, Greek, Japanese, Korean, and Russian; Chinese, Dutch, Greek, Japanese, Korean, and Russian to English; Dutch to French; as well as French to Dutch, Greek, Italian, Portuguese, and Spanish.
    Divides search results into "Web," "Images," "Directory," "Yellow Pages," "News," and "Products" categories.

"ohio legal research"
        About 60 Web pages will be retrieved.
"legal research" +ohio
        About 111,000 Web pages will be retrieved.
"legal research" AND ohio
        About 113,000 Web pages will be retrieved.

Web Metasearch Engines

A Web Metasearch Engine sends your search statement to several search services, receives results, deletes duplicates, and displays results in single list.  This type of Web search service can save aggravation and time, because you only need to know one interface to search and don't need to search multiple search services.  In addition, you can often select which services your search will go to.  However, since a Metasearch Engine sends a search statement to several search services, and search services have different search methods, a complex search statement may not run effectively.

Query Server and Vivisimo are two leading metasearch engines.  Both have the following features:

• Boolean operators "and," "or," and "not" (Vivisimo uses "-" for "not")
• Proximity searching - "near"  [Note:  No current search engines support "near" and this type of searching may no longer work in the metasearch engines.]
• If search engine doesn't support operator (eg, "and," "near"), uses next general operator
• Phrase searching
• Not case sensitive.
• Search nesting within parentheses

"ohio legal research"
       About 60 Web pages, in 8 clusters, will be retrieved.  [About 30 Web pages identified as duplicates.]
"legal research" NEAR ohio
       About 80 Web pages, in 6 clusters, will be retrieved.  [About 30 Web pages identified as duplicates.]
"legal research" AND ohio
       About 80 Web pages, in 6 clusters, will be retrieved.  [About 30 Web pages identified as duplicates.]
 

Vivisimo
    Over 12 search engines, Open Directory, Yahoo, as well as numerous media, government, and business sites.
    Use "advanced search" to run search on selected search engines, news, etc.
    "Clusters" (ie, arranges by subject) results and ranks clusters. Cluster classification not predetermined (as in Query Server); this allows for maximum classification flexibility.
    Supports field searching.

"ohio legal research"
       About 60 Web pages, in 15 clusters (some with sub-clusters), will be retrieved.
"legal research" NEAR ohio
       About 160 Web pages, in 23 clusters (some with sub-clusters), will be retrieved.
"legal research" AND ohio
       About 150 Web pages, in 31clusters (some with sub-clusters), will be retrieved.

Invisible Web

The Web includes a lot of "Dynamic Content."  This "Invisible Web" or "Deep Web" is information transmitted via the Web, rather than stored on the Web in "static" form, and thus is not available for indexing by search engines.  [Robots Exclusion Protocol - Prevents search engine's robot crawler from accessing/indexing portions of Web site.]
The Invisible Web includes information in databases, password restricted information, text within graphics, etc., and is estimated to be 400,000 Web sites.

If you cannot find information on a topic via a Web Search Directory or Search Engine, try an Invisible Web search service.  You may be led to a fee-based Web site, but at least you'll find out if any information is available on your topic.

CompletePlanet
    Over 70,000 searchable databases and search services.
    Can browse sites via its 34 top-level subject "browse tree" (eg, "Government").  ["Browse tree" extends to five sub-levels.]
    Can search within "browse tree" subject directory.
    "Advanced" searching available.

Librarians' Index to the Internet
    Searchable annotated subject directory of over 14,000 Internet resources; librarians select resources based on their "usefulness to users of public libraries."
    Can browse subject directory (eg, "Government & Law").
    "Advanced Search" available.

Additional Resources

Web-Based Articles

Deep Web Research
Marcus P. Zillman
Law Library Resource Xchange
2/23/03

Evaluating the Quality of Information on the Internet
The Virtual Chase, created 9/14/01, revised 9/16/04

Books

Links are provided for items held at the Cleveland-Marshall College of Law Library.

The Extreme Searcher's Guide To Web Search Engines: A Handbook For the Serious Searcher
Randolph Hock, foreword by Reva Basch. - 2nd ed. - Medford, NJ: CyberAge Books, c2001.
[Available in Cleveland-Marshall College of Law Library - Reference ZA4226 .H63 2001]

Government Information On The Internet
Peggy Garvin, ed. - 6th ed. - Lanham, MD: Bernan, a division of Kraus Organization Ltd., c2003.
[Available in Cleveland-Marshall College of Law Library - Reference ZA5075 .G68]

Internet Blue Pages: The Guide To Federal Government Web Sites
Laurie Andriot, comp. - Medford, NJ: Information Today, Inc., c2000, 2001-2002.
[Available in Cleveland-Marshall College of Law Library - Reference ZA5075 .A53 2001-2002]

Internet Power Searching: the Advanced Manual / Phil Bradley. - 2nd ed. - New York, NY: Neal-Schuman Publishers, c2002.

The Invisible Web: Uncovering Information Sources Search Engines Can't See
Chris Sherman and Gary Price. - Medford, NJ: CyberAge Books, c2001.
[Available in Cleveland-Marshall College of Law Library - ZA4450 .S54 2001]

The Librarian's Internet Survival Guide: Strategies For the High-Tech Reference Desk
Irene E. McDermott; edited by Barbara Quint. - Medford, NJ: Information Today, Inc., c2002.
[Available in Cleveland-Marshall College of Law Library - Reference ZA4201 .M36 2002]

Search Engines For The World Wide Web / Alfred and Emily Glossbrenner. - 3rd ed. - Berkeley, CA: Peachpit Press, c2001.

Toward a Cyberlegal Culture / Mirela Roznovschi. - 2nd ed. - Ardsley, NY: Transnational Publishers, c2002.

The United States Government Internet Manual / Peggy Garvin, ed. - Lanham, MD:  Bernan Press, c2004.
[Available in Cleveland-Marshall College of Law Library - Reference ZA5075 .G68 2003-2004]

Web of Deception:  Misinformation on the Internet / Anne P. Mintz, ed. - Medford, NJ:  CyberAge Books, c2002.
[Available in Cleveland-Marshall College of Law Library - Reference ZA4201 .W43 2002]

Annuals and Periodicals

The Internet Lawyer. - Baltimore, MD: Daily Record Company. Monthly.
[Available from Cleveland-Marshall College of Law Library; current issue on Reserve.]

Internet Law & Strategy. - Philadelphia, PA: Law Journal Newsletters, a division of American Lawyer Media. Monthly.
[Available from Cleveland-Marshall College of Law Library; current issue on Reserve.]

The Legal List: Research On The Internet
St. Paul, MN: West Group, Inc. Annual.
[Available from Cleveland-Marshall College of Law Library - KF242 .A1 L375; current edition in Reference]