Comparing Search Results Summaries for Technical Support Documents: A User Study

Pressure to reduce costs has driven many businesses to move technical support from human help desks to the Web. Customers expect to find information easily and quickly when they visit a support site. If they don’t, they may turn to a competitor. This
paper reports on an empirical evaluation of different types of summaries for the presentation of search results on the technical support website of a large computer manufacturer. The summary types tested were: the summaries shown on the current live site, our own rule-based abstracts, a search terms in context summary, and no summary at all (that is, titles only). Although there were no significant differences between summary types on most measures, the overall pattern of results suggested that the rule- based abstracts were better than the titles only. Notably, 30% of the people in the abstract group were often able to select the right document based solely on information contained in the summary. The implications for summarizing technical support documents for search are discussed.

By: Catherine G. Wolf, Sherman R. Alpert, Lev Kozakov, John G. Vergo, Yurdaer N. Doganata, Clare-Marie Karat

Published in: RC22719 in 2003

LIMITED DISTRIBUTION NOTICE:

This Research Report is available. This report has been submitted for publication outside of IBM and will probably be copyrighted if accepted for publication. It has been issued as a Research Report for early dissemination of its contents. In view of the transfer of copyright to the outside publisher, its distribution outside of IBM prior to publication should be limited to peer communications and specific requests. After outside publication, requests should be filled only by reprints or legally obtained copies of the article (e.g., payment of royalties). I have read and understand this notice and am a member of the scientific community outside or inside of IBM seeking a single copy only.

RC22719.pdf

Questions about this service can be mailed to reports@us.ibm.com .