Abstract: Recent works on shared autonomy and assistive-AI technologies, such as assistive robotic teleoperation, seek to model and help human users with limited ability in a fixed task. However, these approaches often fail to account for humans' ability to adapt and eventually learn how to execute a control task themselves. Furthermore, in applications where it may be desirable for a human to intervene, these methods may have inhibited their ability to learn how to succeed with full self-control. In this paper, we focus on the problem of assistive teaching of motor control tasks such as parking a car or landing an aircraft. Despite their ubiquitous role in humans' daily activities and occupations, motor tasks are rarely taught in a uniform way due to their high complexity and variance. We propose an AI-assisted teaching algorithm that leverages skill discovery methods from reinforcement learning (RL) literature to (i) break down any motor control task into teachable skills, (ii) construct novel drill sequences, and (iii) individualize curricula to students with different capabilities. Through an extensive mix of synthetic and user studies on two motor control tasks---parking a car with a joystick and writing characters from the Balinese alphabet---we show that assisted teaching with skills improve student performance by around 40% compared to practicing full trajectories without skills, and practicing with individualized drills can result in up to 25% further improvement.
Overview of our Approach: We train a CompILE model over expert demonstrations (e.g., handwritten Balinese text) for our SKILLEXTRACTOR function, which we use to select a diverse set of scenarios (e.g. words) covering many skills. After a student provides trajectories for these scenarios, we use SKILLEXTRACTOR again to identify their individual expertise for each skill, which informs the set of drills we provide them to practice.
User Study Overview for PARKING:
Walk-through video of a participant's interaction with our teaching interface for the PARKING task, where the goal is to learn how to park the yellow car in the blue square.
User Study Overview for WRITING:
Walk-through video of a participant's interaction with our teaching interface for the WRITING task, where the goal is to learn how to write Balinese characters on the drawing canvas under a time limit.
Individualized Drills from User Study: Our individual skill expertise identification algorithm identifies a wide range of skills as low expertise across students for both environments in our user study. Reverse actions are identified as the most common skills for students to improve on in Parking (right), while common curved-shapes are identified for Writing (left).
Student Trajectories Before & After Individualized Drills Teaching: Students after practicing with individualized drills learn to pay more attention to character details, such as the upwards curve in the bottom right hoop - off-trajectory marks tend to still follow character shapes after teaching.
Our expert behavior for the Parking task. Here, the expert reverses directly until it reaches the front of the spot, and then quickly enters.
Synthetic student trained w/ Behavior Cloning on expert demonstrations but trained with only 50 epochs - the students behavior is erratic, clearly not correct but it is difficult to semantically describe with what skills the student is struggling with.
Synthetic student trained w/ Behavior Cloning on expert demonstrations with limited exposure to reverse actions - the student chooses to move forward, leading to a longer trajectory and therefore worse reward.