CVPR 2023 Workshop on 3D Vision and Robotics

3D perception is critical in robotic applications, such as manipulation and navigation. Understanding the visual world is critical for robots to operate in the real world. In recent years, we have witnessed tremendous progress in deep learning algorithms for processing and making sense of 3D data, such as segmentation and detection. These exciting developments in 3D vision have paved the ground for tackling fundamental challenges in robot perception. Furthermore, connecting 3D vision with robotics will stimulate new research opportunities in active vision, interactive perception, and vision-based decision-making. Nonetheless, a myriad of research challenges and open questions remains. To tackle these challenges, we seek to create a shared forum for interdisciplinary researchers in 3D vision and robotics to share fresh ideas and build new connections.

This workshop will provide a venue for people interested in 3D and robotics to come together to discuss the various challenges and problems in this area. This workshop will accept submissions and focus on related discussion topics such as the ones below:

• Is 3D useful for robotics? What kind of 3D representations are useful for robotics?

• How to imbue the robot with the ability to ground language and concepts in spatial reasoning?

• How can we learn robot perception from human activity data in 3D environments?

• How can robots benefit from implicit neural representations (e.g., NeRF)?

• Both robotics and 3D data are fields that are research areas with high barriers to entry. How can we enable researchers from other fields, such as ML, to work more easily in these areas?

Since our first workshop, which was successfully held at CVPR 2021, we have seen rapidly growing research interests and efforts on related topics in the community. Many new powerful technologies are therefore being invented, such as implicit neural representations for 3D data, neural radiance fields, transformers, large language, visual-lingual models, etc. Many researchers have already demonstrated the use of these latest technologies in solving robotic tasks involving 3D perception. In this second workshop, we invite speakers and paper submissions on these newly trendy topics and technologies, as well as look ahead, discussing the remaining challenges and the potential next steps.

Schedule

08:00am-08:10am: Opening Remarks

08:10am-08:50am: Pete Florence, Google - NeRFing and Embodying Language Models

08:50am-09:30am: Cewu Lu, SJTU - General Robotic Grasping with Human-like Capability and High Learning Efficiency

09:30am-10:00am: Accepted Paper Spotlight Talks (The Morning Sections)

• Understanding 3D Object Interaction from a Single Image
  Shengyi Qian (University of Michigan)*; David Fouhey (University of Michigan)
  PDF

• A Universal Semantic-Geometric Perception Network for Robotic Manipulation
  Tong Zhang (Tsinghua University)*; Yingdong Hu (Tsinghua University); Hang Zhao (Tsinghua University); Yang Gao (Tsinghua University)
  PDF

• Learning to Register Unbalanced Point Pairs
  Kanghee Lee (POSTECH)*; Junha Lee (Postech); Jaesik Park (POSTECH)

• TacSDF: A DeepSDF Approach for 3D Shape Reconstruction Using Vision-Based Tactile Sensing
  Mauro Comi (University of Bristol)*; Alex Church (University of Bristol); Kejie Li (University of Oxford); Laurence Aitchison (University of Bristol); Nathan Lepora (University of Bristol)
  PDF

• Learning Depth Vision-Based Personalized Robot Navigation Using Latent State Representations
  Jorge de Heuvel (University of Bonn)*; Nathan Corral (University of Bonn); Benedikt Kreis (University of Bonn); Bennewitz Maren (University of Bonn)
  PDF Supplementary

• 3D-IntPhys: Towards More Generalized 3D-grounded Visual Intuitive Physics under Challenging Scenes
  Haotian Xue (Georgia Tech); Antonio Torralba (MIT); Joshua Tenenbaum (MIT); Daniel Yamins (Stanford University); Yunzhu Li (Stanford University & University of Illinois at Urbana-Champaign)*; Hsiao-Yu Tung (Carnegie Mellon University)
  PDF

• NeuGraspNet: 6-DoF Grasp Detection in Clutter with Neural Surface Rendering
  Snehal Jauhri (TU Darmstadt)*; Georgia Chalvatzaki (TU Darmstadt); Ishikaa Lunawat (National Institute of Technology Tiruchirappalli)

10:00am-10:30am: Coffee Break & Accepted Paper Posters (The Morning Sections)

10:30am-11:10am: Georgia Gkioxari, Caltech - Towards 3D Object Detection in the Wild

11:10am-11:50am: Saurabh Gupta, UIUC - Understanding and Articulating Articulated Objects

11:50pm-01:00pm: Lunch

01:00pm-01:40pm: Andrew Davison, ICL/Dyson - Real-Time 3D Scene Representation for Robotics

01:40pm-02:20pm: Angel Chang, Simon Fraser University - Towards Intelligent Home Robots

02:20pm-02:50pm: Accepted Paper Spotlight Talks (The Afternoon Sections)

• Reviewing 3D Object Detectors in the Context of High-Resolution 3+1D Radar
  Patrick Palmer (TU Dortmund University); Martin Krueger (TU Dortmund University)*; Richard Altendorfer (ZF); Ganesh Adam (ZF); Torsten Bertram (TU Dortmund University)
  PDF Supplementary

• Distilled Feature Fields Enable Open-Ended Few-Shot Manipulation
  William Shen (MIT); Ge Yang (University of Chicago)*; Alan Yu (MIT); Jansen Wong (MIT); Tomas Lozano-Perez (MIT); Leslie Kaelbling (MIT); Phillip Isola (MIT)
  PDF

• StereoNavNet: Learning to Navigate using Stereo Camera with Auxiliary Occupancy Voxels
  Hongyu Li (Brown University)*; Huaizu Jiang (Northeastern University); Taskin Padir (Northeastern University)
  PDF

• Learning Part-Aware Visual Actionable Affordance for 3D Articulated Object Manipulation
  Yuanchen Ju (Southwest University); Haoran Geng (Peking University); Ming Yang (Beihang University); Yiran Geng (Peking University); Yaroslav Ponomarenko (Peking University); Taewhan Kim (Peking University); He Wang (Peking University); Hao Dong (Peking University)*
  PDF

• Few-View Object Reconstruction with Unknown Categories and Camera Poses
  Hanwen Jiang (UT Austin)*;  Zhenyu Jiang (UT Austin), Yuke Zhu (Kristen Grauman), Yuke Zhu (UT Austin)
  PDF

• Dynamic-Resolution Model Learning for Object Pile Manipulation
  Yixuan Wang (University of Illinois at Urbana-Champaign)*; Yunzhu Li (Stanford University & University of Illinois at Urbana-Champaign); Katherine Driggs-Campbell (University of Illinois at Urbana-Champaign); Li Fei-Fei (Stanford University); Jiajun Wu (Stanford University)
  PDF

• Reassembling Broken Objects using Breaking Curves
  Luca Palmieri (Ca' Foscari University of Venice)*; Ali Alagrami (Ca' Foscari University of Venice); Sinem Aslan (Ca' Foscari University of Venice); Marcello Pelillo (Ca' Foscari University of Venice); Sebastiano Vascon (Ca' Foscari University of Venice & European Centre for Living Technology)
  PDF

02:50pm-03:20pm: Coffee Break & Accepted Paper Posters (The Afternoon Sections)

03:20pm-04:00pm: Xiaolong Wang, UC San Diego - Geometric Robot Learning for Generalizable Dexterous Manipulation

04:00pm-04:40pm: Jitendra Malik, UC Berkeley - Hands and Objects

04:40pm-05:40pm: Panel Discussion

05:40pm-06:00pm: Closing Remarks

Speakers

Call for Abstracts

We solicit 2-4 page extended abstracts (excluding references and supplementary materials) conforming to the official CVPR style guidelines. A paper template is available in LaTeX and Word. References will not count towards the page limit. The review process is double-blind. Submissions can include: late-breaking results, under-review material, archived, or previously accepted work (please make a note of this in the submission).

Submission page: https://cmt3.research.microsoft.com/3DVR2023

Important Dates

• Submission Deadline: March 17, 2023 (11:59 pm CST)

• Reviews Due: April 10, 2023 (11:59 pm CST)

• Acceptance Decision: April 17, 2023 (11:59 pm CST)

• Camera-Ready Version: April 28, 2023 (11:59 pm CST)

Please note the accepted contributions will be presented as spotlight talks in the workshop, and will be posted on the workshop website upon author approval. Accepted papers will not be included in the proceedings of CVPR 2023.

Organizers