Virtual Humans (VH) are artificial interactive characters who look and act like humans, but inhabit simulated environment. One important aspect of VHs is their language understanding and dialogue capabilities. We have created a wide range of role-playing VHs for training, simulation, education, and entertainment. These systems have several aspects in common — they operate in narrow domains, where the character responses have to be carefully controlled; the system language understanding has to be robust to the user’s input; available language data is sparse and difficult to generalize, — for example, it is acceptable that a VH provides misleading information to the user if its role requires so. In this presentation I’ll describe how we use a relevance model approach for constructing effective and efficient language understanding and response selection components for VHs. I’ll summarize experiments comparing RM to both traditional text classification and deep learning approaches. I’ll demo one of our recent VH systems.
Much of human dialogue occurs in semi-cooperative settings, where agents with different goals attempt to agree on common decisions. Negotiations require complex communication and reasoning skills, but success is easy to measure, making this an interesting task for AI. We gather a large dataset of human-human negotiations on a multi-issue bargaining task, where agents who cannot observe each other's reward functions must reach an agreement (or a deal) via natural language dialogue. For the first time, we show it is possible to train end-to-end models for negotiation, which must learn both linguistic and reasoning skills with no annotated dialogue states. We also introduce dialogue rollouts, in which the model plans ahead by simulating possible complete continuations of the conversation, and find that this technique dramatically improves performance.
AI-powered technologies are helping conversations get smarter and more efficient. Building these conversational technologies require a personalized understanding while ensuring a high-level of privacy and security. In order to enable a truly private experience, the intelligence needs to run on the user’s device. On-device intelligence is introducing not only new challenges but also new opportunities. It enables sensing across multiple modalities in real-time which enables experiences that have never been possible before. These capabilities have led to novel technologies that drive these personalized conversational experiences. This talk will cover existing and upcoming technologies that are enabling these magical assistive experiences on our devices.
Recent developments in spoken capabilities and natural (dialog) interactions of conversational search systems, as well as their inclusion in cyber-physical systems, offer new opportunities for adults with intellectual disability to be included in interactive information access. This new opportunity is a result of systems releasing stringent literacy requirements, effectively alleviating the need for abstracting and spelling queries.
I will briefly present and reflect on three case studies of fieldwork, observations and semi-structured interviews with adults with intellectual disability as they interacted with phone-based conversational agents [1], voice activated input for web search systems [2] and a robot equipped with a screen (Pepper).
These three case studies highlight the importance of multimodality, particularly the synergy between audio and visuals, for people with intellectual disability. They also illustrate approaches to respectfully engage with people with intellectual disability for the evaluation of interactive systems [3]. In doing so, they raise the importance of inclusive user studies that can bring to light the diversity of pronunciation or interaction styles that systems need to learn to adapt to.
Deep reinforcement learning (DRL) has been mainly applied to gaming tasks, but it is a promising approach for dialogue management optimisation. In this talk, I will first motivate the need for reinforcement learning in dialogue management and then I will focus on two techniques of deploying DRL that are particularly well suited for dialogue policy optimiation, but also for other interactive learning tasks.
Although algorithmic trading and automation have been topics of significant discussion in finance, a rich variety of social (i.e., linguistic) interaction between humans still drives a large volume of financial transactions in the global capital markets. In the race to gain market advantage, recent advances in natural language understanding are creating new opportunities to assist traders more and more in these financial interactions. In this talk, we will introduce some key concepts for how the global capital markets function, the role of social interactions in trading, as well as use cases for dialogue understanding in this industry.
Intelligent speakers are one of the rapidly evolving areas. I will describe the Alquist social chatbot. Alquist has been twice in a row second in the Amazon Alexa Prize competition. The bot has been developed by a student's team from the Czech Technical University in Prague. I will briefly review the competition rules and its main goal. The main focus will be the Alquist architecture and infrastructure. I will also explain the development of the dialogs for favorite conversational topics. Finally, I will describe some of the key Alquist algorithms.
Even conversational systems have attracted a lot of attention recently, the current systems sometimes fail due to the errors from different components. This talk first focuses on learning language embeddings specifically for spoken inputs in order to mitigate the speech recognition errors. Then we enhance the capability of the current conversational systems so that it can support not only the structured database but also the unstructured texts. These two research directions are highlighted in this talk for guiding the future research potential.