ALOE 2023
Accepted Papers

Spotlights

Voyager: An Open-Ended Embodied Agent with Large Language Models
Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, Anima Anandkumar

OMNI: Open-endedness via Models of human Notions of Interestingness
Jenny Zhang, Joel Lehman, Kenneth Stanley, Jeff Clune

WebArena: A Realistic Web Environment for Building Autonomous Agents
Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, Graham Neubig

Motif: Intrinsic Motivation from Artificial Intelligence Feedback
Martin Klissarov, Pierluca D'Oro, Shagun Sodhani, Roberta Raileanu, Pierre-Luc Bacon, Pascal Vincent, Amy Zhang, Mikael Henaff

Eureka: Human-Level Reward Design via Coding Large Language Models
Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, Anima Anandkumar

Quality Diversity through Human Feedback
Li Ding, Jenny Zhang, Jeff Clune, Lee Spector, Joel Lehman

Posters

t-DGR: A Trajectory-Based Deep Generative Replay Method for Continual Learning in Decision Making
William Yue, Bo Liu, Peter Stone

Procedural generation of meta-reinforcement learning tasks
Thomas Miconi

On the importance of data collection for training general goal-reaching policies.
Alexis D. Jacq, Manu Orsini, Gabriel Dulac-Arnold, Olivier Pietquin, Matthieu Geist, Olivier Bachem

Stackelberg Driver Model for Continual Policy Improvement in Scenario-Based Closed-Loop Autonomous Driving
Haoyi Niu, Qimao Chen, Yingyue Li, Yi Zhang, Jianming HU

Continual Driving Policy Optimization with Closed-Loop Individualized Curricula
Haoyi Niu, Yizhou Xu, Xingjian Jiang, Jianming HU

Rethinking Teacher-Student Curriculum Learning under the Cooperative Mechanics of Experience
Manfred Diaz, Liam Paull, Andrea Tacchetti

Multi-Agent Diagnostics for Robustness via Illuminated Diversity
Mikayel Samvelyan, Davide Paglieri, Minqi Jiang, Jack Parker-Holder, Tim Rocktäschel

Unlocking the Power of Representations in Long-term Novelty-based Exploration
Steven Kapturowski, Alaa Saade, Daniele Calandriello, Charles Blundell, Pablo Sprechmann, Leopoldo Sarra, Oliver Groth, Michal Valko, Bilal Piot

Improving Intrinsic Exploration by Creating Stationary Objectives
Roger Creus Castanyer, Joshua Romoff, Glen Berseth

JaxMARL: Multi-Agent RL Environments in JAX
Alexander Rutherford, Benjamin Ellis, Matteo Gallici, Jonathan Cook, Andrei Lupu, Garðar Ingvarsson, Timon Willi, Akbir Khan, Christian Schroeder de Witt, Alexandra Souly, Saptarashmi Bandyopadhyay, Mikayel Samvelyan, Minqi Jiang, Robert Tjarko Lange, Shimon Whiteson, Bruno Lacerda, Nick Hawes, Tim Rocktäschel, Chris Lu, Jakob Nicolaus Foerster

Toward Open-ended Embodied Tasks Solving
Wei Wang, Dongqi Han, Xufang Luo, Yifei Shen, Charles Ling, Boyu Wang, Dongsheng Li

Does behavioral diversity in intrinsic rewards help exploration?
Aya Kayal, Eduardo Pignatelli, Laura Toni

Diverse Offline Imitation Learning
Marin Vlastelica, Jin Cheng, Georg Martius, Pavel Kolev

SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents
Xuhui Zhou, Hao Zhu, Leena Mathur, Ruohong Zhang, Haofei Yu, Zhengyang Qi, Louis-Philippe Morency, Yonatan Bisk, Daniel Fried, Graham Neubig, Maarten Sap

Skill-Conditioned Policy Optimization with Successor Features Representations
Luca Grillotti, Maxence Faldor, Borja G. León, Antoine Cully

AgentTorch: Agent-based Modeling with Automatic Differentiation
Ayush Chopra, Jayakumar Subramanian, Balaji Krishnamurthy, Ramesh Raskar

Mini-BEHAVIOR: A Procedurally Generated Benchmark for Long-horizon Decision-Making in Embodied AI
Emily Jin, Jiaheng Hu, Zhuoyi Huang, Ruohan Zhang, Jiajun Wu, Li Fei-Fei, Roberto Martín-Martín

Exploration with Principles for Diverse AI Supervision
Hao Liu, Matei Zaharia, Pieter Abbeel

RAVL: Reach-Aware Value Learning for the Edge-of-Reach Problem in Offline Model-Based Reinforcement Learning
Anya Sims, Cong Lu, Yee Whye Teh

Discovering Temporally-Aware Reinforcement Learning Algorithms
Matthew Jackson, Chris Lu, Louis Kirsch, Robert Lange, Shimon Whiteson, Jakob Foerster

SmartPlay : A Benchmark for LLMs as Intelligent Agents
Yue Wu, Xuan Tang, Tom Mitchell, Yuanzhi Li

From Centralized to Self-Supervised: Pursuing Realistic Multi-Agent Reinforcement Learning
Violet Xiang, Logan Cross, Jan-Philipp Fränken, Nick Haber

CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalization
Bodhisattwa Prasad Majumder, Bhavana Dalvi Mishra, Peter Jansen, Oyvind Tafjord, Niket Tandon, Li Zhang, Chris Callison-Burch, Peter Clark

Mix-ME: Quality-Diversity for Multi-Agent Learning
Garðar Ingvarsson, Mikayel Samvelyan, Manon Flageat, Bryan Lim, Antoine Cully, Tim Rocktäschel

CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
Zhibin Gou, Zhihong Shao, Yeyun Gong, yelong shen, Yujiu Yang, Nan Duan, Weizhu Chen

How the level sampling process impacts zero-shot generalisation in deep reinforcement learning
Samuel Garcin, James Doran, Shangmin Guo, Christopher Lucas, Stefano Albrecht

PufferLib: Making Reinforcement Learning Libraries and Environments Play Nice
Joseph Suarez

Training Reinforcement Learning Agents and Humans with Difficulty-Conditioned Generators
Sidney Tio, Pradeep Varakantham

ACES: generating diverse programming puzzles with autotelic language models and semantic descriptors
Julien Pourcel, Cédric Colas, Pierre-Yves Oudeyer, Laetitia Teodorescu

Objectives Are All You Need: Solving Deceptive Problems Without Explicit Diversity Maintenance
Ryan Boldi, Li Ding, Lee Specto

AssemblyCA: A Benchmark of Open-Endedness for Discrete Cellular Automata
Keith Yuan Patarroyo, Abhishek Sharma, Sara Walker, Lee Cronin

Emergence of collective open-ended exploration from Decentralized Meta-Reinforcement learning
Richard Bornemann, Gautier Hamon, Eleni Nisioti, Clément Moulin-Frier

What can AI Learn from Human Exploration? Intrinsically-Motivated Humans and Agents in Open-World Exploration
Yuqing Du, Eliza Kosoy, Alyssa Li Dayan, Maria Rufova, Alison Gopnik, Pieter Abbee

HomeRobot: Open-Vocabulary Mobile Manipulation
Sriram Yenamandra, Arun Ramachandran, Karmesh Yadav, Austin S Wang, Mukul Khanna, Theophile Gervet, Tsung-Yen Yang, Vidhi Jain, Alexander Clegg, John M Turner, Zsolt Kira, Manolis Savva, Angel X Chang, Devendra Singh Chaplot, Dhruv Batra, Roozbeh Mottaghi, Yonatan Bisk, Chris Paxton

Quality-Diversity through AI Feedback
Herbie Bradley, Andrew Dai, Hannah Benita Teufel, Jenny Zhang, Koen Oostermeijer, Marco Bellagente, Jeff Clune, Kenneth Stanley, Gregory Schott, Joel Lehman

Curriculum Learning for Cooperation in Multi-Agent Reinforcement Learning
Rupali Bhati, SaiKrishna Gottipati, Clodéric Mars, Matthew E. Taylor

Vision-Language Models as a Source of Rewards
Harris Chan, Volodymyr Mnih, Feryal Behbahani, Michael Laskin, Luyu Wang, Fabio Pardo, Maxime Gazeau, Himanshu Sahni, Dan Horgan, Kate Baumli, Yannick Schroecker, Stephen Spencer, Richie Steigerwald, John Quan, Gheorghe Comanici, Sebastian Flennerhag, Alexander Neitz, Lei M Zhang, Tom Schaul, Satinder Singh, Clare Lyle, Tim Rocktäschel, Jack Parker-Holder, Kristian Holsheimer

Noisy ZSC: Breaking The Common Knowledge Assumption In Zero-Shot Coordination Games
Usman Anwar, Jia Wan, David Krueger, Jakob Nicolaus Foerster

MCU: A Task-centric Framework for Open-ended Agent Evaluation in Minecraft
Haowei Lin, Zihao Wang, Jianzhu Ma, Yitao Liang

Curriculum Learning from Smart Retail Investors: Towards Financial Open-endedness
Kent Wu, Ziyi Xia, Shuaiyu Chen, Xiao-Yang Liu

minimax: Efficient Baselines for Autocurricula in JAX
Minqi Jiang, Michael D Dennis, Edward Grefenstette, Tim Rocktäschel

Quality Diversity in the Amorphous Fortress: Evolving for Complexity in 0-Player Games
Sam Earle, M Charity, Julian Togelius, Dipika Rajesh

LiFT: Unsupervised Reinforcement Learning with Foundation Models as Teachers
Taewook Nam, Juyong Lee, Jesse Zhang, Sung Ju Hwang, Joseph J Lim, Karl Pertsch

Adaptive Coalition Structure Generation
Lucia Cipolina-Kun, Ignacio Carlucho, Kalesha Bullard

Syllabus: Curriculum Learning Made Easy
Ryan Sullivan

Diversity from Human Feedback
Ren-Jian Wang, Ke Xue, Yutong Wang, Peng Yang, Haobo Fu, Qiang Fu, Chao Qian

Mastering Memory Tasks with World Models
Mohammad Reza Samsami, Artem Zholus, Janarthanan Rajendran, Sarath Chandar

JARVIS-1: Open-world Multi-task Agents with Memory-Augmented Multimodal Language Models
Zihao Wang, Shaofei Cai, Anji Liu, Xiaojian Ma, Yitao Liang

Learning to Act without Actions
Dominik Schmidt, Minqi Jiang

DOGE: Domain Reweighting with Generalization Estimation
Simin Fan, Matteo Pagliardini, Martin Jaggi