Speech Trumps Finger: Examining Modality Usage in a Mobile 3G Environment

As Third Generation (3G) networks emerge they provide not only higher data transmission rates but also the ability to transmit both voice and low latency data within the same session. This paper describes the architecture and implementation of a multimodal application (voice and text) that uses natural language understanding combined with a WAP browser to access email messages on a cell phone. We also report on a field trial that evaluated this system and compared its usage with that of text-only system that is representative of current products in the market today. The paper presents a discussion of the modality usage that was observed during the trial as well as highlighting both implementation problems and usability issues that were encountered in this real world trial. Speech was used the majority of the time by participants for both input and navigation even though most of the participants had little or no prior experience with speech systems (yet did have prior experience with text-only access to applications on their phones). The study confirms the more general previous finding that the multimodal system was preferred to the unimodal interaction, while to our knowledge this represents the first implementation and evaluation of its kind using this combination of technologies on an unmodified cell phone. Design implications resulting from the study findings and usability issues encountered are presented to inform the design of future multimodal mobile applications.