Research Projects

Fault-Aware Robust Control via Adversarial Reinforcement Learning.

Advisior: Huaping Liu (Tsinghua University)

Time: Sep, 2020 - Nov, 2020

Descriptions: Humans and animals can quickly adapt to any damage to their fingers and joints. However, a minor damage could break down an entire robot system. The goal was to increase the robustness of robot over damage in both manipulation tasks and locomotion tasks. Adversarial reinforcement learning methods was implemented. Our algorithm demonstrates significantly increased robustness over damage. It can also be directly applied to real robots without any fine-tune.

Click here to access the pdf file of the paper.

Evolvegraph: Multi-agent trajectory prediction with dynamic relational reasoning.

Advisor: Masayoshi Tomizuka (University of California, Berkeley)

Time: March, 2020 - June, 2020

Descriptions: Multi-agent systems are prevalent in the world, e.g., vehicles in traffic, particles in physical systems. It remains a difficult problem to extract the interaction pattern in such systems and predict the movements of each agent. We proposed a graph-neural-network based algorithm, which captures the dynamically evolving interaction patterns and predict multi-modalities movement of each agent. Our algorithm was tested on H3D dataset, NBA dataset and simulated physical systems and has a superior performance.

Click here to access the paper. The paper was accepted at NeurIPS 2020 conference.

Sample-based Interactive Planning for Urban Autonomous Driving

Advisor: Masayoshi Tomizuka (University of California, Berkeley)

Time: Sep, 2019 - Dec, 2019

Descriptions: The goal is to build up a planner that can interact with other drivers and work in a complex roundabout. In this project, I incorporated Dynamic Bayesian Network and Unscented Kalman Filter to predict intentions of each obstacle car, then sample each possible speed curve, and used Mixture Density Network or Gaussian Mixture Network to predict possibilities of every possible speed curve, finally designed a sample-based planner to select the best one.

Robust Grasp with a Compliant Multi-Finger Hand

Advisor: Oussama Khatib (Stanford University)

Time: Jun, 2019 - Aug, 2019

Descriptions: The goal was to exploit compliance control and grasp objects without force or visual sensors and without disturbing it. By using operational space force control, the hand could make contact with the surface and by probing in the vicinity, we could get the surface normal. Then I combined convex optimization method and greedy search together to get local optimal grasp points and grasp forces. More than 2000 lines of c++ or python code were used in this project.

Here are the demo videos, all the videos are played four times faster.

Music-driven Choreography Based on Machine Learning Methods

Advisor: Changshui Zhang (Tsinghua University)

Time: Nov, 2018 - Jun, 2019

Descriptions: The goal was to use existing videos to train a choreographer to automatically generate a solo dance given a piece of music. I Used openpose lib to extract human posture from videos and used MFCCs signal of music as model input. I Designed a two-layer LSTM with fully connected layers to train the model. Generated dances have different reactions to different tempos.

Videos trained by published dataset.

Videos trained by arbitrary youtube solo dance video.

Operational Space Properties for a Macro-Micro System of UAVs

Advisor: Richard Voyles (Purdue University)

Time: Aug, 2018 - Feb, 2019

Descriptions: I Found that a micro manipulator attached to a UAV (Unmanned Aerial Vehicle) can have better dynamics performance (redueced effective inertia) by using operational space theory. And I Worked out mathematical formulations for building up an operational space hierarchical controller for a UAV with a parallel manipulator, including Jacobian matrix, mass matrix, and null space control. I proposed that using operational space controller, the UAV with a manipulator can have higher control bandwidth.

EMG-based Exoskeletons Control Strategy

Advisor: Chenglong Fu (Tsinghua University)

Time: Jan, 2018 - May, 2018

Descriptions: In order to solve the problem of poor adaption to different gaits of exoskeletons, I invented a strategy using EMG (electromyography) signals to control exoskeletons. Because of the extremely noisy EMG signals and strict limitations on controller design, it's difficult to implement complex algorithm. In my method, spikes in the filtered signals were detected and I added a predefined delay based on gait frequency to get the suiable time to trigger the exoskeletons. The algorithm were tested on the existing exoskeleton in the lab.

Here is the demo video, and the noise in the video means the exoskeleton is triggered.