This work proposes a two-stage reconstruction pipeline on transparent object scenes, based on single-view depth completion + segmentation, and multi-view optimization using Bundle Adjustment and epipolar-assisted optical flow 2D landmark estimation.

This work proposes a category-level object pose estimation network, TransNet, that works on transparent objects in highly challenging scenes. In the work the input modality combination, network architecture, and comparison with recent state-of-the-art is analyzed.

This work proposes a compositional benchmark on robotic manipulation tasks with language instructions. Different from other benchmarks, tasks are categorized by 3D position and 3D orientation constraints, and could be automatically generalized to novel object shapes.

This work proposes a large-scale dataset of daily transparent objects (63 objects, 350K frames), with rgb, depth, normal, mask images and object-level 6D pose annotations. We provided benchmark results of recent depth completion and object pose estimation deep network solutions as well.

This work proposes a rapid customized dataset labeller for objects' 6D poses. The labelled dataset is proved to be more accurate than LabelFusion and synthesized data from Blenderproc with respect to deep CNN pose estimation and pose-based robot grasping.

This work proposes Affordance Coordinate Frame as a novel representation to connect the visual perception and action execution for semantic tasks more than pick and place.

This work incorporates specific feature extraction using light-field sensors with CNNs for better accuracy in transparent material object 6D pose estimation. The estimated poses of glass cups made it possible for robots to build a transparent champagne tower.

This work combines discriminative power of CNNs and robustness from probabilistic inference in object pose estimation in dark environments. In the proposed two-stage pipeline, a particle optimization process takes input from neural network results as prior and sample in SE(3) space to finalize the object pose.

Performing daily interaction tasks such as opening doors and pulling drawers in unstructured environments is a challenging problem for robots. The emergence of soft-bodied robots brings a new perspective to solving this problem. In this paper, inspired by humans performing interaction tasks through simple behaviors, we propose a hierarchical control system for soft arms, in which the low-level controller achieves motion control of the arm tip, the high-level controller controls the behaviors of the arm based on the low-level controller, and the top-level planner chooses what behaviors should be taken according to tasks.