A Semantic Approach to the Dynamic Design of Interaction Controls in Conversation Systems

To support a full-fledged multimedia conversation, we are building an intelligent framework, called Responsive Information Architect (RIA), which can automatically synthesize multimedia responses during a human-computer conversation. As part of its visual response generation, RIA dynamically creates context-sensitive interaction controls that are visual interfaces through which users can further
interact with RIA. To enable systematic design of interaction controls, we study and abstract control properties. In particular, in this paper, we present a semantic model that captures the intentional, presentational, and behavioral characteristics of interaction controls. Using this model, we show how to systematically sketch the semantics of interaction controls.