Agent Learning in Open-Endedness (ALOE) 

CALL FOR PAPERS

Rapid progress in sequential decision-making via deep reinforcement learning (RL) and, more recently, large language models (LLMs) has resulted in agents capable of succeeding in increasingly challenging tasks. However, once the agent masters the task, the learning process typically ends. In contrast, the real world presents endless, novel challenges, which in turn shape the evolution of humans and other organisms that must continually solve them for survival. While so far no artificial learning algorithm has produced an intelligence as general as humans, we know that human intelligence itself resulted from such open-ended co-evolution among agents and the environment. How can we devise learning systems that kickstart and sustain similarly open-ended learning, whereby the learning process generates an endless stream of problems that continually challenge and push further the capabilities of the participating agents? Such open-ended learning (OEL) systems hold the potential to produce agents with increasingly general capabilities, including the ability to succeed in surprising emergent scenarios that might not have been explicitly considered when designing the learning system—leading to improved performance in important settings like sim2real and more broadly, out-of-distribution generalization.

While such OEL agents may seem like an abstract idea, ML models deployed on the web are precisely such agents---including interactive LLMs, which are increasingly used to take direct actions in the world. These deployed models interact with and shape the evolution of their environment, consisting of end users and the web itself, which in turn shape these models’ future training data. Moreso, when the agent is a large generative model, it can directly output its own training data based on what it has currently learned. Despite the recent surge in OEL systems in the wild and in research, such self-fulfilling learning dynamics are still poorly understood.

The 2nd Agent Learning in Open-Endedness (ALOE) Workshop invites researchers to consider OEL systems in the age of large generative models, both in simulation and in the wild:

• How can we better understand, shape, and exploit the potentially open-ended learning dynamics of large generative models in the wild?

• What practical measures of open-endedness are closely aligned with the emergence of new capabilities, and how can we apply them to real-world systems?

• Can we take advantage of substructures in open-ended problem spaces to efficiently train generally-capable agents, for example, through adaptive curricula?

• Can we produce agents that continue to explore and represent knowledge about a world with infinitely rich states and dynamics?

OPEN-ENDED DIRECTIONS

We invite authors to submit papers focused on these and other challenges of learning in open-ended environments. Papers can be up to 9 pages, excluding references and appendices, in either the NeurIPS 2023 format or ICLR 2024 format. In particular, we encourage submissions related to open-endedness in the following areas:

• Benchmarks for open-endedness

• Scalable, open-ended environments and simulations

• Quality-diversity algorithms

• Continual learning

• Curriculum learning / unsupervised environment design

• Emergent complexity

• Self-supervised reinforcement learning

• Multi-agent / population-based / co-evolutionary methods

• Self-organizing systems

• Real-world applications of open-ended learning systems

🌱

PROGRAM COMMITTEE