Reinforcement learning aims to solve sequential decision problems through observations and interactions with the environment, human demonstrations or logged data from historical interactions. It has demonstrated great success in simulated environments and games with clear rules and boundaries, such as Atari Games, Go, Shogi, Poker and many robotic simulations. Such success in these highly complex environments grants promises that reinforcement learning can scale to real world scenarios. Recently, we observe an increasing trend of interest in bringing reinforcement learning to production and optimize beyond immediate consequences in real-life problems. The 1st Reinforcement Learning Ready for Production workshop, held at AAAI 2023, focuses on understanding reinforcement learning trends and algorithmic developments that bridge the gap between theoretical reinforcement learning and production environments. 

Reinforcement Learning for Interactive Robotics

14:30 - 15:30 PM

Abstract: There have been significant advances in the field of robot learning in the past decade. This includes approaches that integrate online and offline reinforcement learning for robotics applications. However, many challenges still remain when considering how robot learning can advance interactive agents, i.e., robots that collaborate with humans such as effectively exploring the space of partner strategies, training with humans in the loop, and designing reward functions. In this talk, I will be discussing the role of learning representations for robots that interact with humans and robots that interactively learn from humans through a few different vignettes. I will first discuss how the notion of latent strategies — low dimensional representations sufficient for capturing non-stationary interactions — can enable reducing the search space of reinforcement learning agents that will effectively coordinate and collaborate with other non-stationary partners such as humans. In addition, we can go beyond partner modeling, and influence or stabilize these latent strategies. These stabilizations can be effective in other coordination settings such as bimanual manipulation. Finally, I will discuss how we can actively learn reward functions that capture human preferences using data-efficient techniques that also allow for expressive neural models. I will end the talk with some ongoing and future directions for leveraging  large language models that can enable designing human preference reward functions.

Bio: Dorsa Sadigh is an assistant professor in Computer Science and Electrical Engineering at Stanford University.  Her research interests lie in the intersection of robotics, learning, and control theory. Specifically, she is interested in developing algorithms for safe and adaptive human-robot and human-AI interaction. Dorsa received her doctoral degree in Electrical Engineering and Computer Sciences (EECS) from UC Berkeley in 2017, and received her bachelor’s degree in EECS from UC Berkeley in 2012.  She is awarded the Sloan Fellowship, NSF CAREER, ONR Young Investigator Award, AFOSR Young Investigator Award, DARPA Young Faculty Award, Okawa Foundation Fellowship, MIT TR35, and the IEEE RAS Early Academic Career Award.