Catálogo de publicaciones - libros

We report on a pilot experiment that investigated the effects of different eye gaze behaviours of a cartoon-like talking face on the quality of human-agent dialogues. We compared a version of the talking face that roughly implements some patterns of human-like behaviour with two other versions. In one of the other versions the shifts in gaze were kept minimal and in the other version the shifts would occur randomly. The talking face has a number of restrictions. There is no speech recognition, so questions and replies have to be typed in by the users of the systems. Despite this restriction we found that participants that conversed with the agent that behaved according to the human-like patterns appreciated the agent better than participants that conversed with the other agents. Conversations with the optimal version also proceeded more efficiently. Participants needed less time to complete their task.

In a multimodal human-machine conversation, user inputs are often abbreviated or imprecise. Simply fusing multimodal inputs together may not be sufficient to derive a complete understanding of the inputs. Aiming to handle a wide variety of multimodal inputs, we are building a context-based multimodal interpretation framework called MIND (Multimodal Interpreter for Natural Dialog). MIND is unique in its use of a variety of contexts, such as domain context and conversation context, to enhance multimodal interpretation. In this chapter, we first describe a fine-grained semantic representation that captures salient information from user inputs and the overall conversation, and then present a context-based interpretation approach that enables MIND to reach a full understanding of user inputs, including those abbreviated or imprecise ones.

The goal of the Conversational Interfaces project at CSLI is to develop a general purpose architecture which supports multi-modal dialogues with complex devices, services, and applications. We are developing generic dialogue management software which supports collaborative activities between a human and devices. Our systems use a common software base consisting of the Open Agent Architecture, Nuance speech recogniser, Gemini (SRI’s parser and generator), Festival speech synthesis, and CSLI’s “Architecture for Conversational Intelligence” (ACI). This chapter focuses on one application of this architecture - an intelligent tutoring system for shipboard damage control. We discuss the benefits of adopting this architecture for intelligent tutoring.

In this chapter we describe the MIAMM project. Its objective is the development of new concepts and techniques for user interfaces employing graphics, haptics and speech to allow fast and easy navigation in large amounts of data. This goal poses challenges as to how can the information and its structure be characterized by means of visual and haptic features, how the architecture of such a system is to be defined, and how we can standardize the interfaces between the modules of a multi-modal system.

It is difficult for a developer to account for all the surface linguistic forms that users might need in a spoken dialogue computer application. In any specific case users might need additional concepts not pre-programmed by the developer. This chapter presents a method for adapting the vocabulary of a spoken dialogue interface at run-time by end-users. The adaptation is based on expanding existing pre-programmed concept classes by adding new concepts in these classes. This adaptation is classified as a supervised learning method in which users are responsible for indicating the concept class and the semantic representation for the new concepts. This is achieved by providing users with a number of rules and ways in which the new language knowledge can be supplied to the computer. Acquisition of new linguistic knowledge at the surface and semantic levels is done using multiple modalities, including speaking, typing, pointing, touching or image capturing. Language knowledge is updated and stored in a semantic grammar and a semantic database.

We describe two major dialogue system segments: the first is an analysis module that learns to assign dialogue acts from corpora, but on the basis of limited quantities of data, and up to what seems to be some kind of limit on this task, a fact we also discuss. Secondly, we describe a Dialogue Manager which uses a representation of stereotypical dialogue patterns that we call Dialogue Action Frames, which are processed using simple and well understood algorithms, which are adapted from their original role in syntactic analysis role, and which, we believe, generate strong and novel constraints on later access to incomplete dialogue topics.