Bringing Robots to the Computer Vision Community

Monday June 17 (PM), Location - 204


There has been a surge of interest in robotics in the vision community. A number of recent papers at CVPR/ICCV/ECCV tackle robotics problems: mobile navigation, visual servoing for manipulation and navigation, visual grasping/pushing, localization, embodied visual-question answering, vision and language navigation, mobility simulators. A number of papers investigate how to effectively study data collected from robots for visual learning. Research also focuses on video analysis for learning affordances. Active vision is also becoming increasingly popular.

While these works demonstrate impressive results, most of them shy away from showing results on real robots, likely because of the lack of expertise in the community, inaccessibility to robotic platforms, and even just the fear of dealing with robotic hardware. This limits the impact of these works, as the robotics community typically believes in results only when they see successful deployments on physical platforms (with good reason). Furthermore, abstracting out the physical system, also removes important and interesting research challenges. Thus, the exposure to physical robots can guide practitioners toward more fruitful research directions and lead to more impactful work.

The goal of this tutorial is to fulfill this gap in expertise and equip interested participants with basic tools that are useful for building, programming and operating robots. We will focus on popular use cases in the community, and use a running example on a open-source low-cost manipulator with a mobile base. We believe this expertise will enable computer vision researchers to better understand perception issues with real robots, demonstrate their algorithms on real systems in the real world, and make it easier for research to transfer between vision and robotics communities.




  1. Hardware (Dhiraj Gandhi)
  2. Software and Controls (Adithya Murali, Saurabh Gupta)
  3. Coffee Break
  4. Learning and testing on real robots (Lerrel Pinto)
  5. Case Studies