Speech Trumps Finger: Examining Modality Usage in a Mobile 3G Environment

As Third Generation (3G) networks emerge they provide not only higher data transmission rates but also the ability to transmit both voice and low latency data within the same session. This paper describes the architecture and implementation of a multimodal application (voice and text) that uses natural language understanding combined with a WAP browser to access email messages on a cell phone. We also report on a field trial that evaluated this system and compared its usage with that of text-only system that is representative of current products in the market today. The paper presents a discussion of the modality usage that was observed during the trial as well as highlighting both implementation problems and usability issues that were encountered in this real world trial. Speech was used the majority of the time by participants for both input and navigation even though most of the participants had little or no prior experience with speech systems (yet did have prior experience with text-only access to applications on their phones). The study confirms the more general previous finding that the multimodal system was preferred to the unimodal interaction, while to our knowledge this represents the first implementation and evaluation of its kind using this combination of technologies on an unmodified cell phone. Design implications resulting from the study findings and usability issues encountered are presented to inform the design of future multimodal mobile applications.

By: Jennifer Lai, Stella J. Mitchell, Chris Pavlovski

Published in: RC23359 in 2004


This Research Report is available. This report has been submitted for publication outside of IBM and will probably be copyrighted if accepted for publication. It has been issued as a Research Report for early dissemination of its contents. In view of the transfer of copyright to the outside publisher, its distribution outside of IBM prior to publication should be limited to peer communications and specific requests. After outside publication, requests should be filled only by reprints or legally obtained copies of the article (e.g., payment of royalties). I have read and understand this notice and am a member of the scientific community outside or inside of IBM seeking a single copy only.


Questions about this service can be mailed to reports@us.ibm.com .