Title: Multimodal analysis and synthesis for conversational research

Abstract:

In this talk, we will discuss three research threads. We will present the general trends, followed by our contributions. First thread involves technologies for off-line multimodal assessment and feedback in interviewing and presentation settings. While nonverbal and verbal assessment is important from the interviewer's view-point, feedback is important for the candidate. Recent trends in terms of research will be summarized, that includes both modeling and HCI developments. The second research thread concerns technologies for real-time multimodal conversational agents and applications. Multimodal analysis enables understanding user emotion and state, multimodal dialog enables making use of this information along with spoken text and subsequently generate the surface text along with the appropriate state for the virtual agent, and finally the multimodal synthesis module generates suitable nonverbal behavior and prosody for the agent reply. These systems can find applications in healthcare or education, where information can be solicited from the user and a simple task-oriented conversation can be accomplished. Innovations for quickly customizing the avatar appearance and animations are also emerging. The final research thread involves Generative Adversarial Networks (GAN) based multimodal analysis and synthesis systems to perform controlled conversational research. GANs can generate human centered images and videos, and hence enabling study of appearance and behavior manipulation, which enables controlled conversational experiments.

Bio:

Dr. Dinesh Babu Jayagopi is currently an Associate Professor at IIIT Bangalore, where he heads the Multimodal Perception Lab. His research interests are in Audio-Visual Signal Processing, Machine Learning, and Social Computing. He obtained his doctorate from Ecole Polytechnic Federale Lausanne (EPFL), Switzerland, beginning of 2011. He received the Outstanding paper award in the International Conference on Multimodal Interaction (ICMI), 2012, Idiap PhD student research award for the year 2009. He also received the Indian Department of Science and Technology (DST) Young Scientist Start up Grant in 2016. He has successfully collaborated with Defence Research and Development Organization, Openstream AI, Accenture Labs and NI Systems. He was a visiting professor at University of Lausanne in summer 2019.

Title: The point of action where cognitive behavioral therapy is effective

Abstract:

Cognitive behavioral therapy (CBT) is a treatment method that eases feelings and reduces stress by working on the way of thinking and receiving (cognition) of things, and also on some behavior. This is one of the effective psychotherapy performed for depression and anxiety disorders in the actual clinical situation of psychiatry. CBT includes cognitive reconstruction, behavioral activation, and problem-solving techniques. What is the point of action of CBT as a treatment?

Creatures gather various information from their surroundings and recognize them, that is, process them and connect them to appropriate actions. In the process of the evolution, in order to obtain rapid action, we acquired feelings like an instant "signal system" that indicate the result of cognition, and put the process up to cognition below consciousness. In other words, the "signal system" i.e. feelings lights up instantly, and the actions that accompany it can be swiftly taken. However, if the information gathering that leads to cognition is not successful, it is assumed that cognition would be false and that it would lead to false "signals", that is, false feelings. It is assumed that information is narrowed in depression and cause a depressed mood. By optimizing this information gathering process, CBT guides correct cognition and optimizes feelings. This is the point of action of CBT.

Our research team is working on the development of automated CBT using AI based on the point of action of this CBT.

Bio:

Takashi Kudo was born in Osaka, Japan, in 1958. He received the M.D. degree from Osaka Medical College in 1986, and the Ph.D. degree from the Graduate School of Medicine, Osaka University, in 1990. From 1990 to 1992, he holds a post-doctoral position with the New York State Institute Basic Research in Developmental Disabilities. He was an Assistant Professor in 1994–1999 and an Associate Professor in 1999–2013, respectively, with the Department Psychiatry, Graduate School of Medicine, Osaka University. Since 2013, he has been a Professor with the Department of Mental Health Promotion, Graduate School of Medicine, Osaka University.