Abstract

Non-prehensile manipulation is pivotal for object relocation, especially in scenarios like a smart restaurant where robots are tasked with relocating dishware. Past research aimed at deriving analytic solutions for stable pushing, yet these efforts necessitate knowledge of the object's physical attributes like friction coefficient, center of friction, shape, and mass distribution to achieve a viable solution. Addressing this, we introduce a vision-based algorithm for stable tableware push planning. This algorithm processes the top-view depth image of unknown tableware, samples the push contact, and determines the stable push direction for each sampled contact, subsequently engaging in stable push path planning.

This innovative approach allows for the successful relocation of dishware to desired locations without any prerequisite knowledge of the objects' physical properties. Through this method, we can enhance the efficiency and effectiveness of robotic tasks in various practical applications, marking a step forward in autonomous object relocation and handling.

Analytic Boundary of Stable Pushing

We define stable pushing as keeping the slider (object being pushed) fixed to the pusher (object that pushes) throughout the pushing task. Assuming line-contact, quasi-static pushing, stable planar pushing is affected by both the friction cones by the pusher and the slider's friction center [1], as shown in Fig 1. The green-colored region is the region between two perpendicular lines to each friction cone of the push contact [2] , while the red-colored region indicates the region between a tip line [3] and a bisector between the center of friction and push contact. 

Vision-Based Stable ICR Region Estimation Algorithm

Initially, the object  Fig 2 (a) is imaged from a top-view to obtain a depth image  Fig 2 (b), following which edge points are identified using Sobel filtering. Based on the gripper's width, push contact candidates are then sampled  Fig 2 (c). The illustrations depict normal vectors of the push contact with red arrows, while the blue arrow represents the pushing direction.

Subsequently, for each push contact candidate, the stable ICR region is determined  Fig 2 (d). The visual representation distinguishes between stable and unstable ICR regions using green and red coloring, respectively. This systematic approach enables the calculation of stable left-hand and right-hand ICR regions for each push contact on the given object. By distinguishing between stable and unstable ICR regions, the methodology aids in better understanding and planning the pushing mechanics to achieve desired object relocation, especially in scenarios where traditional grasping is not viable or efficient.

Hybrid A* Algorithm-based Stable Push Planning

The integration of the hybrid A* algorithm in stable push planning facilitates the robot in relocating the dish to a specified position without necessitating re-approach during the pushing activity. Initially, candidate push contacts are sampled in accordance to a pushing direction segmented into 45° increments. Following this, the left and right stable Instant Center of Rotation (ICR) regions are computed for every push contact, and the radius of the ICR nearest to the origin is identified among the ICRs where non-holonomic motion is viable.

The push planning algorithm is then employed to determine the maximal pushing steering angle from the minimal ICR radius, which serves to avert the object from slipping away from the gripper during the pushing action. Utilizing this steering angle, stable push paths are calculated for the candidate push contacts. Subsequently, the shortest path along with the corresponding push contact is selected to ensure an efficient and effective object relocation strategy. This methodical approach underpins a robust framework for planning and executing stable pushing tasks, enhancing the robot's capability in manipulating and relocating objects in a controlled and precise manner.

Experiment and Result

The efficacy of the vision-based stable push planning algorithm was assessed through a physical pushing experiment conducted on 'N' tableware of varying shapes, as depicted in Fig 3. The scene was captured using a structured light camera (Zivid Two), and the pushing actions were carried out by a parallel-jaw gripper (Robotiq 2f-85) mounted on a Doosan Robotics M1013 manipulator, as illustrated in Figure 2. Success in pushing was defined by the object's relocation within the designated goal region.

The summarized outcomes, presented in Table 1, affirm the algorithm's effectiveness in handling smaller objects, albeit its performance dwindles with larger objects. The future trajectory of this work is geared towards enhancing the push path planning to adeptly manage larger objects. This enhancement aims at incorporating alterations in the contact position with the least number of changes, alongside adapting the algorithm to cater to 3D objects. This includes an analysis of potential toppling, which is an additional complexity when transitioning from planar to three-dimensional objects. Through these proposed advancements, the objective is to broaden the algorithm's capability, ensuring reliable and effective push planning across a diverse array of object sizes and dimensions.

Demonstration

We demonstrated the stable push path generating module. From Fig 5 and Fig 6, we can observe that the robot can stably push the dish according to the stable push path given from the module.

References

[1] Lynch, K.M.: Nonprehensile robotic manipulation: Controllability and planning. Carnegie Mellon University (1996) 

[2] Mason, M. T. (1986). Mechanics and planning of manipulator pushing operations. The International Journal of Robotics Research, 5(3), 53-71. 

[3] Peshkin, M., & Sanderson, A. (1986, April). Manipulation of a sliding object. In Proceedings. 1986 IEEE International Conference on Robotics and Automation (Vol. 3, pp. 233-239). IEEE.