The agenda and timing of individual sessions are subject to change.

Morning Program

• 9:00 AM — Opening Remarks

• 9:10 AM — Invited Talk 1: Intelligence in Action
  Speaker: Prof. Özgür Şimşek

Abstract: Reinforcement learning algorithms aim to learn how to select actions so that reward received from the environment can be maximized in the long term. In doing this, the general approach is to work with a static action space, as received from the system designer at the beginning of learning. The enrichment of the action space through learning remains relatively unexplored to date. In this talk, we will explore some recent ideas on how a reinforcement learning agent can autonomously augment its action space by creating new and useful temporally-extended actions that allow efficient exploration, learning, and generalisation.

Bio: Özgür Şimşek is a professor at University of Bath where she leads the Artificial Intelligence Research Group. From 2018 to 2020, she served as Deputy Director at IMI. Before joining the University of Bath in 2017, Özgür was a research scientist at the Center for Adaptive Behaviour and Cognition at the Max Planck Institute for Human Behaviour in Berlin, Germany. She received her PhD in Computer Science in 2008 from the University of Massachusetts Amherst.  Özgür’s research spans a broad range of areas in machine learning, including reinforcement learning, supervised learning, learning from small data sets, and bounded rationality. 

• 9:30 AM — Invited Talk 2: Surveys in RL (and RLHF)
  Speaker: Prof. Serena Booth

Abstract: This talk explores how survey methodology can improve reinforcement learning (RL) and reinforcement learning from human feedback (RLHF). Drawing on our recent FAccT and TMLR papers, I will discuss two complementary directions: redesigning how human feedback is elicited through improved survey framing, and using surveys to develop more accurate models of human decision-making.

Bio: Serena Booth is an Assistant Professor of Computer Science at Brown University. Her research focuses on human-AI interaction, reinforcement learning, and AI alignment, with an emphasis on modeling human decision-making to improve how AI systems learn from people. She received her Ph.D. from MIT and previously served as an AI Policy Advisor in the U.S. Senate. She is the recipient of an NSF CAREER Award, and she is appointed as a Senior Fellow at Microsoft AI Economy Institute.

• 9:50 AM — Invited Talk 3: Sequences of Frames: The Analogy with Memory Retrieval
  Speaker: Prof. Sara Aronowitz

Abstract: We sometimes think of the challenge of framing a problem as a meta-problem that happens before the first-order problem of choosing what to do. But in many cases, it seems like framing (and re-framing) is done in a way that overlaps with first-order planning, both in time and even in terms of processing. In this talk, I use a comparison between the frame problem and the problem of relevance in memory search to ask: what changes if we think of framing as a sequence?

Bio: Sara is an associate professor of philosophy at the University of Toronto, and an affiliate of the Cognitive Science program. She works on learning and memory, and how long timescales change questions about rationality.

• 10:10 AM — Invited Talk 4: Does RL have a blind spot for Knightian uncertainty?
  Speaker: Dr. Joel Lehman

Abstract: RL has been highly effective in settings where the possibilities can be simulated or laid out in advance. Our world is less tidy, continually bubbling up situations nobody could reasonably anticipate, and handling such situations with grace is an important facet of general intelligence. This talk argues that such uncertainty — what economists call Knightian uncertainty, or unknown unknowns — may sit outside RL's formalisms. It further sketches how biological evolution, which manages the unforeseen without formalisms, might inspire new ideas for integrating Knightian uncertainty into RL.

Bio: Joel Lehman is a Principal Research Scientist at Lila Sciences and a Senior Research Associate at the University of Oxford’s Human-Centered AI Lab. His research focuses on reinforcement learning, open-endedness, evolutionary computation, and the development of more humane AI systems. He is co-author of the pop-science book Why Greatness Cannot Be Planned.

• 10:30 AM — Coffee Break

• 11:00 AM — Panel Discussion with Invited Speakers

• 12:00 PM — Lunch and Breakout Sessions

Afternoon Program

• 1:30 PM — Poster Session 1

• 2:30 PM — Contributed Oral Presentations (15min each)

  • Hidden Tuning Is Misleading the RL Community  

Presenter: [Esraa Elelimy]  .  [View Paper] 

• • Toward Enactive Artificial Intelligence  

Presenter: [Banafsheh Rafiee]  .  [View Paper]

• • Generalised Bellman Recurrence and Three Dualities of Sequential Decision-Making 

Presenter: [Fernando Rosas]  .  [View Paper]

• • Frameless Agentic System Theory 

Presenter: [Roy Fox]  .  [View Paper]

• 3:30 PM — Surprise art performance: Dylan Brenneis, Human-Machine Co-creation 

• 3:45 PM — Closing Remarks

• 4:00 PM — Poster Session 2

• 5:00 PM — End of Workshop