Talks

Schedule

9:00 | Opening

9:05 | Talk 1: Alberto Dalla Libera

9:45 | Talk 2: Konstantinos Chatzilygeroudis

10:30 | Coffee break

11:00 | Talk 3: Maryam Kamgarpour

11:45 | Talk 4: Diego Romeres

12:30 | Lunch break

13:30 | Talk 5: Mario Zanon

14:15 | Talk 6: Roberto Calandra

15:00 | End of workshop

Speakers

MC-PILCO: Data-Efficient Model-Based Reinforcement Learning

Reinforcement Learning algorithms proved a promising solution to solve complex control problems in robotics applications. Model-free RL (MFRL) algorithms are particularly effective in settings where a large amount of data is available, such as virtual environments. As an example, MFRL algorithms reach super-human performance in playing Chess, Shogi, and Go. On the contrary, when dealing with physical systems, the number of samples available is limited, possibly compromising the effectiveness of these technologies. This motivates the interest in data-efficient RL algorithms for physical systems. Among the others, Model-Based RL (MBRL) algorithms are a promising solution: by learning and updating a mathematical model of the system based on interaction data, MBRL algorithms limit interaction time on the actual system. In this talk, we present MC-PILCO, a data-efficient MBRL algorithm that relies on Gaussian Process Regression (GPR) to model the system dynamics.

Short bio

Alberto Dalla Libera received a Laurea degree in Control Engineering and the Ph.D. degree in information engineering from the University of Padova, Padua, Italy, in 2015 and 2019, respectively. He is currently a research fellow at the Department of Information Engineering of the University of Padova. His research interests include Robotics, Reinforcement Learning, Machine Learning, and Identification. In particular, he is interested in the application of Machine Learning techniques for modeling and control of physical systems.

Autonomous Skill Discovery and Learning

Developing robots capable of autonomous, open-ended lifelong learning remains one of the most ambitious challenges in robotics. In this talk, we introduce a novel pipeline designed to empower robots to autonomously discover and learn new skills through incremental trial-and-error learning. This approach integrates deterministic simulations, Variational Auto-Encoders, and Reinforcement Learning to generate goal-conditioned policies tailored to a robot's capabilities. By employing effective exploration strategies, the pipeline learns meaningful goal representations and distributions, enabling the efficient acquisition of new skills and enhancing the robot's adaptability in dynamic environments. Then, we delve into methods that seamlessly blend model-based controllers with data-driven approaches to tackle the transferring of learned policies from simulation to the real world (Sim2Real). These methods aim to bridge the Sim2Real gap by preserving stability guarantees while leveraging the flexibility and adaptability of data-driven learning. We highlight examples of how these techniques are applied to robotic systems, including 7DOF manipulators, differential drive and quadruped robots, demonstrating their potential for robust, scalable, and efficient real-world deployments.

Short bio

Konstantinos Chatzilygeroudis received the Integrated Master degree (Engineering Diploma) in computer science and engineering from the University of Patras, Patras, Greece, in 2014, and the Ph.D. degree in robotics and machine learning from Inria Nancy-Grand Est, France and the University of Lorraine, Nancy, France in 2018. From 2018 to 2020 he was a Postdoctoral Fellow with the LASA Team with the Swiss Federal Institute of Technology Lausanne (EPFL), Lausanne, Switzerland. He is a recipient of an H.F.R.I. Grant for Post-doctoral Fellows (2022-2024): he is the Principal Investigator of the project ”Novel Optimization Methods for Autonomous Skill Learning in Robotics” that is being implemented within the Department of Mathematics, University of Patras, Greece. He has also taught and is still teaching several undergraduate and post-graduate courses on Artificial Intelligence, Computer Science and Robotics at University of Patras, Greece. He has also co-supervised several undergraduate and master theses. He is currently serving as an Associate Co-Chair of the IEEE Technical Committee on Model-based Optimization for Robotics, while he has served as an Associate Editor for several years at the International Conference on Intelligent Robotics (IROS), actively participated in the organization committee (as a Chair responsible for the virtual part of the conference) of the International Conference on Robot Learning (CoRL) 2021, and served as an Area Chair for NeurIPS 2024. His work has been published in top-tier journals and conferences in the fields artificial intelligence, machine learning and robotics, and he has received a Best Paper Award at GECCO 2022. He has also actively collaborated with industrial partners: he was the Leader of the R&D Computer Vision Team at Metargus, a pre-seed funded startup (based in Patras, Greece), and he was the Lead Robotics Engineer at Ragdoll Dynamics (company based in London, UK). His research interests include the area of artificial intelligence and focus on reinforcement learning, fast robot adaptation, evolutionary computation and autonomous skill discovery.

Towards Incorporating Safety in Reinforcement Learning

Safety and reachability are fundamental problems in robotic systems control. The stochastic reach-avoid framework developed in our community characterizes the optimal solution to these problems. However, this framework is model-based and generally computationally intractable. In this talk, I will present our work towards a learning-based approach to address the problem. Specifically, I will present our policy gradient reinforcement learning approach to solve the stochastic optimal control problem subject to stochastic reach-avoid constraints. I will outline the technical insights behind the convergence proof of the algorithm, along with its sample complexity for a given accuracy. The talk will conclude by highlighting applications and open challenges.

Short bio

Maryam Kamgarpour holds a Doctor of Philosophy in Engineering from the University of California, Berkeley and a Bachelor of Applied Science from University of Waterloo, Canada. Her research is on safe decision-making and control under uncertainty, game theory and mechanism design. Her theoretical research is motivated by control challenges arising in intelligent transportation networks, robotics, power grid systems and healthcare. She is the recipient of NASA High Potential Individual Award, the European Union (ERC) Starting Grant, and the 2024 European Control Award.

Reinforcement Learning for Human-Robot Collaborative Assembly

Blending human dexterity with robotic speed and precision offers immense potential for small- and medium-sized enterprises engaged in high-mix and low-volume production scenarios. Furniture assembly, with its long action sequences, dexterous handling, precise alignment, and transport of bulky components, provides a challenging yet promising test bed. In this talk, I will outline core challenges in building a comprehensive human-robot collaborative assembly system, along with how reinforcement learning can help tackling these challenges. In particular, I will present three critical components required to achieve this goal: a method for reasoning about the subtasks required to complete the entire assembly; a proactive, reinforcement learning-based robot planner that adapts in real time to human interactions; and improved robot skills designed to gradually shift more responsibility to the robot during complex tasks.

Short bio

Diego Romeres received his M.Sc. degree (summa cum laude) in control engineering and the Ph.D. degree in information engineering from the University of Padua, Padua, Italy, in 2012 and 2017, respectively. He is currently a Senior Principal Research Scientist and Team Leader of the Intelligent Robotics Team at Mitsubishi Electric Research Laboratories, Cambridge, MA, USA. He held visiting research positions at TU Darmstadt, Darmstadt, Germany, and at ETH, Zurich, Switzerland. His research interests include robotics, artificial intelligence, machine learning, reinforcement learning, bayesian optimization and system identification theory.

Safe, Stable and Explainabile Reinforcement Learning

RL is a very successful approach to optimal control, which, however, struggles to provide explainability and strong guarantees on the behavior of the resulting control scheme. In contrast, MPC is a standard tool for the closed-loop optimal control of complex systems with constraints and limitations and benefits from a rich theory to assess closed-loop behavior. Because of model inaccuracy, however, MPC can fail at delivering satisfactory closed-loop performance. This seminar will discuss how to leverage the advantages of the two techniques, offering a path toward safe and explainable RL.

Short bio

Mario Zanon received the Master's degree in Mechatronics from the University of Trento, and the Diplôme d'Ingénieur from the Ecole Centrale Paris, in 2010. After research stays at the KU Leuven, University of Bayreuth, Chalmers University, and the University of Freiburg he received the Ph.D. degree in Electrical Engineering from the KU Leuven in November 2015. He held a Post-Doc researcher position at Chalmers University until the end of 2017. In 2018 he moved to the IMT School for Advanced Studies Lucca where he became Associate Professor in 2021. His research interests include optimal control and estimation of nonlinear dynamic systems, in particular for aerospace and automotive applications, economic MPC, numerical methods for optimization, and reinforcement learning.

Deep Reinforcement Learning in the Real World

Using Deep Reinforcement Learning on physical robots is often a daunting challenge. In this talk, I will discuss several paradigms that allow to approach the learning of control policies for real robots: from self-supervised learning to autonomously scale data collection, to model-based reinforcement learning to improve the data-efficiency, and finally sim2real to leverage existing simulations.

Short bio

Roberto Calandra is a Full (W3) Professor at the Technische Universität Dresden where he leads the Learning, Adaptive Systems and Robotics (LASR) lab. Previously, he founded at Meta AI (formerly Facebook AI Research) the Robotic Lab in Menlo Park. Prior to that, he was a Postdoctoral Scholar at the University of California, Berkeley (US) in the Berkeley Artificial Intelligence Research (BAIR) Lab. His education includes a Ph.D. from TU Darmstadt (Germany), a M.Sc. in Machine Learning and Data Mining from the Aalto university (Finland), and a B.Sc. in Computer Science from the Università degli Studi di Palermo (Italy). His scientific interests are broadly at the conjunction of Robotics and Machine Learning, with the goal of making robots more intelligent and useful in the real world. Among his contributions is the design and commercialization of DIGIT -- the first commercial high-resolution compact tactile sensor, which is currently the most widely used tactile sensor in robotics. Roberto served as Program Chair for AISTATS 2020, as Guest Editor for the JMLR Special Issue on Bayesian Optimization, and has previously co-organized over 16 international workshops (including at NeurIPS, ICML, ICLR, ICRA, IROS, RSS). In 2024, he received the IEEE Early Academic Career Award in Robotics and Automation.

Page updated

Google Sites

Report abuse