Different kinds of challenges summarised in this article. (a): Environment-level challenges that include cluttered environment, extreme lighting environment, and dynamic environment; (b): Object-level challenges that include deformable objects, articulated objects, and grasped objects