Weekly Gathering for

Vision Researchers and Enthusiasts

11:00 am- 12:00 pm, Star Room, 32-D463

Click this to join mail list for receiving the latest talk information. Contact Bolei (bzhou@csail.mit.edu) for scheduling a talk.


Perception for manipulation

Speaker: Peter K.T. Yu from MIT MCube Lab

Time: 11:00 am to 12:00 pm

Date: Tuesday, March 21, 2017

Location: 32-D463


Our team (MIT MCube Lab) participated in Amazon Picking Challenge (APC) in 2015 and 2016, and received second and third place, respectively. The challenge was to develop a fully autonomous system to pick and place products in a warehouse setting, where various products may coexist inside a single bin. Robots are still far from achieving human speed and reliability. In the first half of the talk, I will describe our approach to the APC, list the lessons we have learned, and then propose our wish list regarding technologies that will be useful for developing similar systems in the future.

One major item in the wish list is to exploit physics and contact sensing in the perception system, where vision usually plays a big role but is still insufficient in practice. This can be due to occlusions, inaccurate models, ambiguous appearances, etc. I will spend the second half of the talk on our effort on using pushing mechanics and contact sensing to estimate shape and pose of an object. I will draw the connections between our problem and the SLAM (simultaneous localization and mapping) problem and show how we can apply frameworks developed in the SLAM community. Our results show that incorporating extra information can improve the accuracy from vision alone. Moreover, even when vision cues are temporarily missing, our system can still reason about the object state during manipulation.


Peter is a fourth-year PhD student of Electrical Engineering and Computer Science. He received the degrees of B.S. in Computer Science from National Chiao-Tung University in 2010, and M.S. in Computer Science from National Taiwan University in 2012. He is now working with Prof. Alberto Rodriguez, and Prof. John Leonard in the MCube lab. His research focuses on incorporating contact sensing and physics to improve robot perception. His paper on dataset collection for pushing manipulation was nominated for Best Paper Award in IROS 2016.

Designing robots to improve people's lives has always been his career goal. He worked on perception problems in the DARPA Robotics Challenge (2013-2015), and served as perception and software lead in the Amazon Picking Challenge (Second Place 2015, Third Place in 2016). His website: http://people.csail.mit.edu/peterkty/


Seminar Co-chairs:

Bolei Zhou and Katie Bouman

Bolei Zhou

More resources: MIT Vision Group, MIT CSAIL