Multimodal Interfaces and Interactions
This tutorial is cancelled. We apologise for the inconvenience.
Aim
Multimodal human-computer interfaces as an area of research
interest has gained considerable momentum over the last couple of decades.
These interfaces are inspired by human to human communication wherein
a number of modalities such as speech, gestures, head movements and gaze
patterns are used to improve the bandwidth and robustness of communication.
Multimodal human-computer interfaces being developed today attempt to use – separately or in concert - various user input modes beyond the traditional keyboard and mouse, such as speech, pen, touch, hand gestures, device gestures, gaze and head and body movements. They may also employ one or more output modalities such as haptic /tactile, speech, non-speech and spatial audio in addition to visual output.
As computing becomes more ubiquitous and pervasive than ever
before, multimodal interfaces will play an important role in making interaction
with increasingly invisible computer systems, simple and intuitive.
They also have the potential to enable those with specific impairments,
the
non-tech savvy and those unfamiliar with the desktop paradigm, to
interact
with computing systems.
The focus for the workshop will be to provide an overview of the
key topics and the state of the art in the space of multimodal interaction,
and focus on the design of these interactions, with a special focus
on mobile devices and interactions.
Content
The tutorial will enable participants develop an understanding
and appreciation of multimodal interaction in terms of key concepts ,
the capabilities of the different modalities, state of the art techniques
and open research problems.
Specific topics will include
• introduction to interaction modalities: audio (speech and non-speech),
haptic/tactile and gestural interfaces, and text input.
• overview of existing work in these topics and a glimpse into new
work being carried out,
• approaches to research in the multimodal interaction space
• multimodal fusion
• design and evaluation of socially acceptable multimodal interactions
• A hands-on design session may also be included based on the number
of participants.
Morning
• Introduction to the day
• Interaction modalities: human capabilities and technology with specific
examples of technology and interactions
o Audio (speech and non-speech sound), speech technology)
o Haptics and touch
o Gestures
o Smell and taste
o Novel forms of text input (pen and keyboard)
o Multimodal Fusion
Afternoon
• Designing multimodal interactions
• Designing socially acceptable multimodal interactions
• Group design exercise
o Designing a gesture based interaction
o Testing the interaction in the wild
o Report back from the groups
• The future of multimodal interaction and wrap up
Who should attend?
This tutorial will be of interest to students, researchers and
practitioners in HCI, Computer Sciences and Interaction and Interface
Design.
A short biography of the tutors
Stephen Brewster:
A Professor of Human-Computer Interaction in the Department of Computing
Science at the University of Glasgow since
2001. His research focuses on multimodal interaction, or using multiple
sensory modalities (particularly hearing, touch and smell) to create
richer interactions between human and computer. His work has a strong
experimental focus, applying perceptual research to practical situations.
He has shown that novel use of multimodality can significantly improve
usability in a wide range of situations, for mobile users, visually-impaired
people, older users and in medical applications.
Sriganesh Madhvanath: Sriganesh ("SriG") is a Senior Research Scientist and the Principal Investigator for the Intuitive Multimodal and Gestural Interaction (IMaGIn) project at HP Labs India. This multidisciplinary project aims to explore intuitive touch and visual gesture-based interaction experiences for personal systems, and create technologies to support them. SriG joined HP Labs India in April 2002, where he has led research into handwriting recognition and linguistic resources for Indic scripts (partially open sourced as Lipi Toolkit), standards such as W3C InkML for platform-neutral representation of digital ink and related toolkits, and pen-based interfaces and solutions relevant to the emerging markets in spaces such as form filling, text input and collaboration. Prior to joining HP, SriG was a senior staff engineer with Narus, a startup in Palo Alto, California. Earlier he was Research Staff Member with the Document Analysis and Recognition group at IBM Almaden Research Center. He holds a PhD and MS in Computer Science from the State University of New York at Buffalo and a B.Tech. in Computer Science and Engineering from Indian Institute of Technology, Mumbai.
Rama Vennelakanti: A Senior User Researcher with HP Labs India since 2004. She has a Masters Degree in Social Anthropology from the Andhra University. While at HP Labs India, she has been involved with numerous projects as a user researcher, user experience designer and has led the technology trials for TVPrintCast. Rama has over 16 years of experience of user research, and prior to joining HP Labs India has worked with BPL Mobile as a Senior Manager Market Research, Probe Qualitative Research – IMRB as an Associate Project Director, among other organisations. Her current research interests include studying human to human and human to device interactions in the context of making computing interfaces less intimidating and more natural for the Intuitive Multimodal and Gestural Interaction (IMaGIn) project at HP Labs India.
Back to Workshops & Tutorials
