Seminars
2024
Weds 27th March 1pm, Robotarium Seminar room (G50): Mohan Sridharan "Back to the Future of Cognition and Control in Robotics"
Abstract: In this talk, I will describe my vision and philosophy for designing an architecture for integrated knowledge representation, reasoning, control, and learning in robotics. I will begin by describing the underlying fundamental representational choices, processing commitments, and cognitive principles and theories that allow us to leverage the complementary strengths of knowledge-based and data-driven methods. I will then illustrate the capabilities of the architecture in realistic simulation environments and on physical robots. I will do so in the context of key visual scene understanding, manipulation, embodied AI, and multiagent collaboration problems.
Bio: Prof. Mohan Sridharan is a Chair in Robot Systems in the School of Informatics at the University of Edinburgh (UK). Prior to his current appointment, he held academic positions at the University of Birmingham (UK), The University of Auckland (NZ), and at Texas Tech University (USA). He received his Ph.D. from The University of Texas at Austin (USA). His research interests include knowledge representation and reasoning, cognitive systems, and interactive learning, as applied to robots and agents collaborating with humans. He is also interested in developing algorithms to promote automation and sustainability in domains such as transportation, agriculture, and climate informatics. Web page: https://homepages.inf.ed.ac.uk/msridhar/
Weds 13th March, 1pm, room 9, National Robotarium, first floor, Sandro Pezzelle "From Word Representation to Communicative Success: Beyond Image-Text Alignment in Language-and-Vision Modeling"
Abstract: By grounding language into vision, multimodal NLP models have a key advantage over purely textual ones: they can leverage signals in one or both modalities and potentially combine this information in any way required by a given communicative context. This ranges from representing single words taking into account their multimodal semantics [1] to resolving semantically underspecified image descriptions [2] to adapting their way of referring to images to achieve communicative success with a given audience [3]. Moving from word-level semantics to real-life communicative scenarios, I will present work investigating the abilities of current language and vision models to account for and deal with semantic and pragmatic aspects of human multimodal communication. I will argue that these abilities are necessary for models to successfully interact with human speakers.
[1] Pezzelle, S., Takmaz, E., Fernández, R. (2021). Word Representation Learning in Multimodal Pre-Trained Transformers: An Intrinsic Evaluation. TACL.
[2] Pezzelle, S. (2023). Dealing with Semantic Underspecification in Multimodal NLP. ACL 2023.
[3] Takmaz, E., Brandizzi, N., Giulianelli, M., Pezzelle, S. and Fernández, R. (2023). Speaking the Language of Your Listener: Audience-Aware Adaptation via Plug-and-Play Theory of Mind. Findings of ACL 2023.
Bio: Assistant Professor in Responsible AI at the ILLC, Faculty of Science, University of Amsterdam. Affiliated with the Dialogue Modelling Group. His research combines Natural Language Processing (NLP), Computer Vision, and Cognitive Science, and focuses on multimodal language understanding and generation, behavioral and mechanistic interpretability, and the cognitive mechanisms underlying human semantics. He published in ACL, EACL, EMNLP, NAACL, TACL, Cognition, and Cognitive Science. He is a member of the ELLIS society, a faculty member of the ELLIS Amsterdam Unit, and a board member of SigSem, the ACL special interest group in computational semantics.
Tues March 12th, 1pm, room: CMS01, Stefan Ultes: "Towards Natural Behaviour of Dialogue Systems with Explicit Dialogue Control"
Abstract: The goal of dialogue system researchers has always been to create artefacts that offer natural interaction capabilities and effortlessly communication by means humans also used to communicate among themselves. Even though systems like ChatGPT are already very good in form and style, there are more things to natural dialogue system behaviour than these LLM-based agents are capable of. I believe that this requires additional control capabilities of a dialogue system. In this talk, I will motivate this with insights from an analysis of communication styles in dialogues. I will continue with focussing on work on learning an explicit dialogue control component through reinforcement learning by optimizing on the estimated user satisfaction and thus ultimately improving the perceived naturalness of the interaction. I will finish with arguing that this basic idea is still relevant in the age of LLMs.
Bio: Stefan Ultes is a full professor of natural language generation and dialogue Systems at the Otto-Friedrich-University of Bamberg, Germany, and a member of the executive board of the Bamberg Center for Artificial Intelligence (BaCAI). Previously, he was leading the speech technology research group at Mercedes Benz Research & Development in Sindelfingen, Germany, and a research associate at the spoken dialogue systems group at the University of Cambridge, UK.