Interactive Learning for NLP means training, fine-tuning or otherwise adapting an NLP model to inputs from a human user or teacher. Relevant approaches range from active learning with a human in the loop, to training with implicit user feedback (e.g. clicks), dialogue systems that adapt to user utterances, and training with new forms of human input. Interactive learning is the converse of learning from datasets collected offline with no human input during the training process.
Existing methods often treat the user as a human in the loop who is limited to the role of an oracle. Current NLP research in this vein focuses on techniques such as active learning [1,2], learning from demonstration [3], and preference learning [4]. Relatively little research has considered end users choosing the agent's curriculum or machine teaching (e.g., [5]). A major challenge is adapting large neural models that are expensive to train to a continual stream of user feedback. An emerging body of work has begun to explore user feedback via natural language explanations [6], advice [7] and instructions [8]. Interpreting such inputs requires relating natural language to the world in which the system operates [9]. GPT-3 [10] also allows new tasks to be specified via text interaction, presenting new opportunities for interactive NLP research. While most approaches for dialogue systems do not from learn from the user's utterances, recent research also investigates conversational methods for eliciting preferences [11], user intent [12] and language acquisition [13].
Evaluation with real users is costly, making clear and consistent best practices crucial, along with user simulations. However, current simulated user models assume random errors in user feedback, which is often unrealistic [14]. Explainability and interpretability also play key roles in helping users to understand the effects of their feedback on the model's behaviour [15]. In summary, there are many promising threads of research that address different aspects of interactive systems but leave many open questions. There is now a great opportunity to draw these threads together to develop NLP systems that learn continually from user interaction.