Preference-based learning:
Akrour, R., Schoenauer, M., and Sebag, M. Preference-based policy learning. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 2011.
Akrour, R., Schoenauer, M., and Sebag, M. April: Active preference learning-based reinforcement learning. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 2012.
Wilson, A., Fern, A., and Tadepalli, P. A Bayesian approach for policy learning from trajectory preference queries. In Advances in Neural Information Processing Systems, 2012.
Wirth, C. and Fürnkranz, J. Preference-based reinforcement learning: A preliminary survey. In ECML/PKDD Workshop on Reinforcement Learning from Generalized Feedback: Beyond Numeric Rewards, 2013.
Wirth, C., Fürnkranz, J., and Neumann, G. Model-free preference-based reinforcement learning. In Conference on Artificial Intelligence, 2016.
Sadigh, D., Dragan, A. D., Sastry, S., and Seshia, S. A. Active preference-based learning of reward functions. In Robotics: Science and Systems, 2017.
Biyik, E. and Sadigh, D. Batch active preference-based learning of reward functions. In Conference on Robot Learning, 2018.
Biyik, E., Huynh, N., Kochenderfer, M. J., and Sadigh, D. Active preference-based gaussian process regression for reward learning. In Robotics: Science and Systems, 2020.
Christiano, P. F., Leike, J., Brown, T., Martic, M., Legg, S., and Amodei, D. Deep reinforcement learning from human preferences. In Advances in Neural Information Processing Systems, 2017.
Kimin Lee, Laura Smith, Pieter Abbeel. PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training, ICML 2021
Preference-based RL for dialog / summarization
Sugiyama, H., Meguro, T., and Minami, Y. Preference learning based inverse reinforcement learning for dialog control. In Conference of the International Speech Communication Association, 2012.
Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, Paul Christiano. Learning to summarize from human feedback, 2020
COACH:
MacGlashan, James, Ho, Mark K., Loftin, Robert Tyler, Peng, Bei, Roberts, David L., Taylor, Matthew E., and Littman, Michael L. COACH: Interactive learning from policy-dependent human feedback. In ICML, 2017.
Dilip Arumugam, Jun Ki Lee, Sophie Saskin, Michael L. Littman. DeepCOACH: Deep Reinforcement Learning from Policy-Dependent Human Feedback. 2019
TAMER:
Bradley Knox and Peter Stone. Interactively Shaping Agents via Human Reinforcement. K-CAP 2009.
Garrett Warnell, Nicholas Waytowich, Vernon Lawhern, Peter Stone. Deep TAMER: Interactive Agent Shaping in High-Dimensional State Spaces, AAAI 2018
Learning a Goal Classifier
Avi Singh, Larry Yang, K. Hartikainen, C. Finn, S. Levine. End-to-End Robotic Reinforcement Learning without Reward Engineering, 2019.
Laura Smith, Nikita Dhawan, Marvin Zhang, Pieter Abbeel, Sergey Levine. AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videos, RSS 2020
Annie Xie, Avi Singh, Sergey Levine, Chelsea Finn. Few-Shot Goal Inference for Visuomotor Learning and Planning, CoRL 2018.
Other
Pilarski, P. M., Dawson, M. R., Degris, T., Fahimi, F., Carey, J. P., and Sutton, R. S. Online human training of a myoelectric prosthesis controller via actor-critic reinforcement learning. In International Conference on Rehabilitation Robotics, 2011.
Leike, J., Krueger, D., Everitt, T., Martic, M., Maini, V., and Legg, S. Scalable agent alignment via reward modeling: a research direction. arXiv preprint arXiv:1811.07871, 2018.