Projects

Embodied ChatGPT

Invited demonstration at MSI Robot Block Party 2023

Embed GPT into a real robot to perform actions according to human commands as well as answer questions about the physical world
Robust to new langauge and new object arrangments

dark_brown_wallet_demo.mov

Human Pose Mimicking with Baxter Arms

Invited demonstration at MSI Robot Block Party 2022

Detects human pose with OpenPose
Establishes the mapping between human and Baxter geometries to perform human movements on Baxter

New video by Jiading Fang

Solving Rubik's cube with Baxter

Invited demonstration at MSI Robot Block Party (video1)(video2)

Developed strategies for robots to handle certain transformation of cube faces
Worked on manipulation of cube with robotic hands (parallel jaws)

New video by Xlarity

Delicate Manipulation of Egg with Force Control

Robotic Dynamics Course Project at University of Illinois

Implement force control, position control, path planning on robots to go through obstacles and zigzags and finally push the egg on the plate. The program is coded using C++ with ROS environment.

Demo_VIdeo.MOV

Computer Vision Aided Hanoi Tower

Intro to Robotics Course Project at University of Illinois

Playing the classical 3-layer Hanoi Tower Game using computer vision with forward kinematics. The images are captured by camera on board and are analyzed with color-level segmentation.

20160910_170822[1].mp4

Simulated Autonomous Driving

Udacity autonomous Driving Nanodegree

Develop a deep convolutional neural network to output control actions with raw images as inputs. The dataset was collected using the recordings of the expert driving on simulator and specifically designed to recover from off road.

run3.mp4