We solicit original paper submissions highlighting the challenges preventing RL being actionable in large scale real-life settings. We encourage contributions that shed light on aspects including (but not limited to):
Offline Settings or Limited Exploration: RL typically requires exploration to learn optimal policies, which may not be feasible in offline settings where historical data is fixed and exploration is not possible. Without exploration, RL struggles to discover optimal actions and may fail to generalize to unseen states.
Small Data Settings: RL algorithms often require large amounts of data to learn effectively, making them ill-suited for scenarios with limited data. In such settings, RL models may overfit to the available data or fail to learn meaningful patterns, leading to poor performance.
RL as Inference: RL can be viewed as a form of inference where the agent learns to infer the best actions based on observed states and rewards. However, this inference process can be challenging, particularly in complex environments with high-dimensional state spaces or stochastic dynamics.
Sparse Rewards in Long Horizons: RL struggles with sparse reward signals, especially in tasks with long time horizons where rewards are infrequent or delayed. In such cases, RL agents may struggle to learn effective policies due to the lack of feedback, leading to slow learning or suboptimal solutions.
Non-Unique Solutions in Inverse Problems: Inverse reinforcement learning, where the goal is to infer the underlying reward function from observed behaviour, can suffer from non-uniqueness of solutions. This ambiguity can make it difficult to accurately recover the true reward function, leading to uncertainty in learned policies.
Sample Complexity Issues with Model-Based RL: Model-based RL approaches often require significant amounts of data to learn accurate models of the environment dynamics. High sample complexity can hinder the practical applicability of these methods, especially in real-world settings where data collection may be expensive or time-consuming.
Distributional Shift: RL algorithms are sensitive to changes in the distribution of states or rewards between training and deployment environments. Distributional shift can arise due to changes in the environment dynamics or the introduction of new tasks, leading to poor generalisation and degraded performance of RL agents.
Addressing these challenges requires the development of novel algorithms and techniques that can effectively handle limited data, sparse rewards, uncertainty, and distributional shifts, thus making RL more robust and applicable to a wider range of real-world problems.
Additionally, we welcome submissions that reflect on "what went wrong" for previously completed work (published or unpublished), including postmortem discussions of beautiful ideas that didn't quite pan out. As a community, we seek to fully embrace circumstances where we are inclined to say, "I can't believe it's not better!" In this workshop we aim to champion introspective discussion around failure modes that come about from extending RL to real-world problems, from which we believe there are tangible insights to be gained.
Submission Guide:
Limit of 4 content pages (acknowledgements, references and, appendices not included)
Note that reviewers are not obligated to look at the appendix. The main text must stand on its own.
To format your paper, please use the official LaTeX style files for RLC 2024 (overleaf style files or zip file ).
Code submission, as supplementary material, is highly encouraged
All parts of a submission, including any code, should be anonymized.
The submission should be a single pdf including the references and appendices.
The submission and reviewing process is hosted on OpenReview (deadline: 10 May 2024, Anywhere on Earth).
https://openreview.net/group?id=rl-conference.cc/RLC/2024/Workshop/ICBINB
There will be no rebuttal phase. Authors will be notified of acceptance/rejection after the reviewing phase (31 May 2024).
If the submitted work has previously appeared in a journal, workshop, or conference, the submission should extend or critically address that previous work. Any overlapping published work at the time of submission should be stated.
Non-Archival Policy:
Submissions will not be indexed or have archival proceedings. We welcome submissions of unpublished or ongoing work that will be under review at archival venues such as NeurIPS.
Accepted papers will be displayed on the workshop website after 31st May 2024.
There are three “Best Abstract” awards for submitted paper contributions that are of the highest scientific quality and promote unique ideas that are best representative of the highlighted themes above.
Call for Papers: 29 March 2024
Submission Deadline: 25 May 2024
Reviewing period: 25 May - 31 May 2024
Notification date (papers): 31 May 2024
Workshop Date: 9 August 2024