Information Retrieval and Ranking on the Web: Benchmarking Studies I

The exponential growth of information available on the
World Wide Web has been documented in numerous studies.
The studies also indicate that Internet users are
turning to search engines and search services in
increasing numbers to find the information they are
seeking, but they are not necessarily satisfied with
their performance. Specific problems which have been
cited in user surveys include the speed of transmission
and retrieval of information and the format for
presenting the results from searches. In this report,
we describe some of the components of a new Web-based
search and retrieval system prototype, which is part of
a larger information outlining and visualization system
for Web documents. Our system is based on a modified
version of latent semantic indexing, the output from
which is used to rank the relevance of Web pages for an
input query. We report the speeds of computation of the
singular values and ranking when Householder
bidiagonalization and Givens rotations are used to
compute the singular values of a matrix representation
of document-query space. In particular, our discussions
will emphasize mathematical algorithms, efficient
dynamic memory allocation procedures, and relevance
ranking; linguistic techniques will be discussed only as
necessary for the sake of completeness.