A Methodology for Developing Multimodal User Interfaces of Information Systems

The Graphical User Interface (GUI), as the most prevailing type of User Interface (UI) in today's interactive applications, restricts the interaction with a computer to the visual modality and is therefore not suited for some users (e.g., with limited literacy or typing skills), in some circumstances (e.g., while moving around, with their hands or eyes busy) or when the environment is constrained (e.g., the keyboard and the mouse are not available). In order to go beyond the GUI constraints, the Multimodal (MM) UIs apear as paradigm that provide users with great expressive power, naturalness and flexibility. In this thesis we argue that developing MM UIs combining graphical and vocal modalities is an activity that could benefit from the application of a methodology which is composed of: a set of models, a method manipulating these models and the tools implementing the method. Therefore, we define a design space-based method that is supported by model-to-model colored transformations in order to obtain MM UIs of information systems. The design space is composed of explicitly defined design options that clarify the development process in a structured way in order to require less design effort. The feasability of the methodology is demonstrated through three case studies with different levels of complexity and coverage. In addition, an empirical study is conducted with end-users in order to measure the relative usability level provided by different design decisions.