We are working on using algorithmic robotics to improve the automation and customization in everyday life and in industry.

Collaborative Robotics for Progressive Manufacturing Automation

Robotics offers a promising solution to alleviate the pressing challenge of shrinking labor forces in manufacturing while improving productivity and quality. However, abruptly replacing human workers with robots would be very challenging, because industrial robots are not optimized with a human-replaceable skill set and the high monetary, technological and social barriers for overhauling an existing labor-intensive production line with robotic and automation technologies. Given this reality, we propose the progressive collaborative robotics paradigm to smoothly transit between complete reliance on human workers and full automation: humans and robots should coordinate and complement each other for common tasks in a semi-structured factory environment. The workload of robots could be adjusted according to their intelligence levels towards a fully automated production line. Collaborating with humans calls for a much higher level of complexity from a robot, comparing to the robot working in isolation and following a set of pre-programmed instructions to perform a repetitive task. In this project, we will tackle the three key problems for the collaborative paradigm: 1) guaranteeing safe human-robot interactions; 2) enabling robots to adapt to human workers’ preferences to seamlessly accomplish a joint task; 3) improving the teamwork efficiency via effective task scheduling among robots and workers.

Below are some of our preliminary works on human-robot handover and collaboration:

Deformable Object Grasping and Manipulation

Manipulating deformable objects is a fundamental problem in robotics. For instance, from folding a bed sheet in the bedroom to tying suture in the surgical operating room, the ability to grasp and manipulate deformable objects is an important capability for assistive and autonomous robots to poses. Reliable and efficient robotic manipulation are also significant from an industrial and economic view, since flexible materials are found in almost every industrial product. Our goal is a combination of many techniques, including robotics, perception, computational geometry, and machine learning, to enable precise and intelligent manipulation of deformable objects. Below are some of our recent work along this line:

We also collected a large dataset of knot tying, which is used to improve the accuracy of learning from demonstration. The data set is also available.

To accomplish visual feedback for objects with significant self-occlussion due to self-deformation, we are also working on the simultaneous tracking and reconstruction for robotic manipulation of soft objects.

Multi-agent Systems

High-speed, low-latency obstacle avoidance that is insensitive to sensor noise is essential for enabling multiple decentralized robots to function reliably in cluttered and dynamic environments. While other distributed multi-agent collision avoidance systems exist, these systems require online geometric optimization where tedious parameter tuning and perfect sensing are necessary. We are working on a novel end-to-end framework to generate reactive collision avoidance policy for efficient distributed multi-agent navigation. Our method formulates an agent's navigation strategy as a deep neural network mapping from the observed noisy sensor measurements to the agent's steering commands in terms of movement velocity. Below are some of our recent work along this line. More results are available in our project webpage.

Flexible and Robust Sequential Robotic Manipulation

To rearrange and interact with a scene, a robot needs to be able to grasp and manipulate objects. One of the most common manipulation tasks is the pick-and-place, where the robot picks up a target object at an initial pose and then places it at a target pose. Sometimes the desired target pose is not reachable directly, due to the robot’s kinematics constraints or collisions between the robot and its surrounding environment. A typical solution is the pick-and-place regrasp, i.e., the robot uses a sequence of pick-ups and place-downs to incrementally change the object’s pose. In particular, after the object is picked up by the first grasp, it is stably placed in an intermediate location and then picked up again using another grasp. There are many open problems about sequential robotic manipulation, including how to compute the optimal regrasp sequence, how to perform fast computation, how to design suitable mechanism for the gripper and for the support structure so as to maximize the manipulation capability. Below are some of our recent work along this line:

The manipulation technique is also useful in many industrial applications, such as automated warehouse using robots. Below is the video for the system attending the Amazon Picking Challenge 2015. Some video about the amazon picking challenge work, when the Hong Kong Financial Secretary visited the lab. (full report)

Motion Planning, Trajectory Generation, and Fast Collision Checking