Learning a new motor skill is a complex process that requires extensive training and practice. Several theories from motor learning, neuroscience, education, and game design suggest that curriculum-based training may be the key to efficient skill acquisition. However, traditional methods for designing such training curricula often result in time-consuming, costly, and potentially ineffective motor skill learning. Systematizing and automating the curriculum generation process may improve humans' motor skill learning process. This work is a stepping stone towards the long-term goal of automating the curriculum generation process for human motor skill learning.
Recent advances in artificial intelligence have introduced curriculum learning using reinforcement learning, which has enabled impressive speed-ups in artificial agents’ abilities to learn complex tasks. This thesis draws its inspiration from a two-stage hierarchical model of curriculum learning consisting of two learning agents: a student agent that learns a given task and a \teacher agent that learns the optimal curriculum for training the student agent. The core idea of this thesis is to bring the two-stage curriculum learning approach to design a curriculum for human motor skill acquisition. To accomplish this, we must replace the student agent with a model of human learning, which poses three main challenges: (1) it is not straightforward to accurately represent humans skill level or state of knowledge; (2) unlike artificial agents, limits exist on human training time and repetitions; and (3) human learning cannot be paused or externally controlled. In this thesis, we address these challenges by creating an artificial representation of human motor learning behavior.
Our model of human motor learning is developed in the context of a specific motor task called Reach Ninja. We first model Reach Ninja as a Markov decision process (MDP) to enable RL agents to learn the Reach Ninja task. Using human demonstrations, we then identify the necessary constraints to limit the performance of the RL agent on the Reach Ninja MDP, which brings the learning behavior close to that of humans. The resultant approximate model demonstrates pre-training and post-training performance similar to that of humans. We then design a static curriculum capable of effectively training the artificial agent in our approximate model and test whether the same static curriculum can induce a similar learning behavior in humans. Preliminary tests with human subjects show that training with the same static curriculum did not improve learning efficiency compared to training directly on the target task. Finally, we discuss the methodology for learning a dynamic curriculum based on our model of Reach Ninja and human motor learning.
Link To Thesis: https://drive.google.com/file/d/1EOrcr8o9UMxklmwPE3545_DahyhGj4yw/view?usp=sharing
Link to Reach Ninja Environment Demo: https://www.youtube.com/watch?v=pBFOo35QclQ&t=1s
Robocup is a world-renowned robot soccer competition that envisions defeating a team of human players using robots by the year 2050. I was part of the Austin-Villa Robot Soccer Team at UT Austin which participates in Standard Platform League of RoboCup. The competition uses Nao Humanoid Robots for playing soccer.
During my time at UT, I mainly contributed towards dynamic stabilization of kicking motion and developing new motion behaviors for Robocup 2021. Our team secured 4th position in the competition.
Results Page: https://spl.robocup.org/results-2021/
This project is a replication study of the popular DPG-SLAM algorithm for mapping low dynamic environments. The algorithm enables the robot to remain localized in an environment that changes substantially over time. Using incremental smoothing and mapping (iSAM) as the underlying SLAM state estimation engine, the Dynamic Pose Graph evolves over time as the robot explores new places and revisits previously mapped areas. Laser scans for the same portion of the environment at different times are compared to perform change detection; when sufficient change has occurred in a location, the dynamic pose graph is edited to remove old poses and scans that no longer match the current state of the world. Experimental results are shown for two real-world dynamic indoor laser data sets, demonstrating the ability to maintain an up-to-date map despite long-term environmental changes.
The map in ash color represents the raw pose graph slam output. The map in blue/purple represents the active map with blue points representing the static points and purple points representing the newly added points. Similarly, the red/green map represents the dynamic map with the red points representing the removed map points and green representing the newly added points.
This project is an application of Modular Inverse Reinforcement Learning to achieve human-like optimal game-play on Atari game - Freeway