Learning for intervention

Scenario definitions

Scenario 1: Grasping Object: black-box Approach: Top

Scenario 1: Grasping Object: black-box Approach: Lateral

Scenario 1: Grasping Object: Anphora Approach: Top

Scenario 1: Grasping Object: Anphora Approach: Lateral

Scenario 2: Pushing Object: black-box Approach: Lateral 1

Scenario 2: Pushing Object: black-box Approach: Lateral 2

Scenario 2: Pushing Object: Anphora Approach: Lateral 2

3D Reconstruction Experiments

The robot arm performs a linear movement (joint q2) to produce a laser sweep. A virtual camera captures the scene and a 3D reconstruction algorithm creates a 3D reconstruction of the scene (Point Cloud Data).

Generated Point Cloud: [output.pcd]

Teleoperation Experiments

• • Scenario configuration:

    • the robot is initially placed at a fixed position P1 (but can also be teleoperated in x,y,z)

    • an object (i.e. a black-blox) is laying on the floor and can interact dynamically with other objects (physics engine)

    • the arm can be teleoperated in joint space (q0, q1, q2, q3, q4, q5)

  • the user gets visual feedback from the simulator (the 3D scene + contact information)

  • contact information is received now in binary form (true/false) on the screen. True indicates that there is a contact between the robot and any other object (i.e. the black-box).

  • For the teleoperation, the user uses a gamepad.

  • During the teleoperation and training stage, the system stores all the information in a bagfile. Then, a Matlab file with the proper variables {Timestamp, q0, q1, q2, q3, q4, q5, Contact} is automatically generated for posterior processing.

Experiment 01

Variables: {Timestamp, q0, q1, q2, q3, q4, q5, Contact}

Matlab file: 001.m

Graphical representation

TRITON project

• • 2 underwater validation scenarios:

    1. (1) Black-box (flight recorder) grasping & recovery

    2. (2) Panel intervention: connect / disconnect / turn a valve

  • website: http://www.irs.uji.es/triton/

  • IRSLab YouTube channel: http://www.youtube.com/user/IRSLab

Vehicle: Girona500

• • Ribas, D.; Palomeras, N.; Ridao, P.; Carreras, M.; Mallios, A., "Girona 500 AUV: From Survey to Intervention," Mechatronics, IEEE/ASME Transactions on , vol.17, no.1, pp.46,53, Feb. 2012 [doi]

Robotic arm: Lightweight ARM5E

• • 4 D.O.F. + 1 Gripper

• • [Fernández12] J. J. Fernández, M. Prats, J. C. García, R. Marín, P.J. Sanz and A. Peñalver. “Manipulation in the Seabed: A New Underwater Robot Arm for Shallow Water Intervention”, 1st IFAC Conference on Embedded Systems, Computational Intelligence and Telematics in Control (CESCIT), pp. 314-319 ISBN: 978-3-902661-97-5, 3-5 April 2012, Würzburg, Germany. [on-line]

  • [Fernández13] J. J. Fernández, M. Prats, P. J. Sanz, J. C. García, R. Marin, M. Robinson, D. Ribas, P. Ridao, "Manipulation in the Seabed: A New Underwater Robot Arm for Shallow Water Intervention", in IEEE Robotics and Automation Magazine, to appear in 2013.

Underwater Simulator (UWSim)

• • Website: http://www.irs.uji.es/uwsim/

• • Prats, M.; Perez, J.; Fernandez, J.J.; Sanz, P.J., "An open source tool for simulation and supervision of underwater intervention missions", 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2577-2582, 7-12 Oct. 2012 [doi]

  • ROS interfaces

  • Simulated TRITON scenarios:

Sensors / system inputs

• • Arm joint values: Slew, Shoulder, Elbow, JawRotate (4 D.O.F.)

  • Arm gripper: JawOpening

  • End-effector cartesian position + orientation: (x,y,z,r,p,y)

  • JR3 90M31 force sensor [details]

  • Pan/tilt camera ECA-DTR100ZC [details]

  • Stereo Camera VIDERE STH-MDCS [details]

  • Tactile sensors Tekscan FlexiForce [details]

  • Tritech SeaStripe laser for 3D reconstruction [details]

Control action / system outputs

• • Arm joint values: Slew, Shoulder, Elbow, JawRotate (4 D.O.F.)

  • Arm gripper: JawOpening

  • End-effector cartesian position + orientation: (x,y,z,r,p,y)

User interfaces / feedback

• • Multimodal user interfaces. (Not yet implemented on UWSim)

• • • [Garcia10] J. C. García, J. J. Fernández, P. J. Sanz, R. Marín, Increasing Autonomy within Underwater Intervention Scenarios: The user interface approach, 4th Annual IEEE International System Conference, pp. 71-75, San Diego (USA), April-2010. [doi]

    • [Marin02] R. Marín, P.J. Sanz, A.P. del Pobil. “Teleoperated Robot System Via Web: The UJI Telerobotic Training System”. International Journal of Robotics and Automation, Special Issue on Web-based Robotics and Automation. Volume 17/Number 3/2002, paper 105, ACTA Press, 2002.

• • On-screen feedback information (not implemented)

  • Gamepad (cartesian / articular spaces) (not configured on simulator)

  • SMI iView eye tracker [specs]

    • Diego R. Faria, Ricardo Martins, Jorge Lobo, Jorge Dias, Extracting data from human manipulation of objects towards improving autonomous robotic grasping, Robotics and Autonomous Systems, Volume 60, Issue 3, March 2012, Pages 396-410, ISSN 0921-8890. [online]

Procedures to learn

• • Robot end effector plug connector on the valve (following the apropiate orientation)

  • Robot end effector unplug connector from the valve

  • Robot end effector grasps the black-box in a stable manner

Learning techniques

• • Supervised learning (bayesian approach)

  • Learning by demonstration

  • Reinforcement learning (?)

Related references

• • [Prats13] M. Prats, Á. P. del Pobil, P. J. Sanz, “Robot Physical Interaction through the combination of Vision, Tactile and Force Feedback”, Springer Tracts in Advanced Robotics, Volume 84 2013. ISBN: 978-3-642-33240-1 (Print) 978-3-642-33241-8 (Online). http://dx.doi.org/10.1007/978-3-642-33241-8.

  • [Prats12a] M. Prats, D. Ribas, N. Palomeras, J. C. García, V. Nannen, S. Wirth, J. J. Fernández, J. P. Beltrán, R. Campos, P. Ridao, P. J. Sanz, G. Oliver, M. Carreras, N. Gracias, R. Marín, A. Ortiz. “Reconfigurable AUV for intervention missions: a case study on underwater object recovery”,. Intelligent Service Robotics, Volume 5 (1), pp. 19-31, January 2012. [link]

  • [Prats12b] M. Prats, J. J. Fernández, P. J. Sanz, “Combining Template Tracking and Laser Peak Detection for 3D Reconstruction and Grasping in Underwater Environments”, Proceedings of the 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 106-112, Vilamoura, Algarve (Portugal). 978-1-4673-1735-1/12/S31.00. , October 2012 [doi]

  • [Sanz12] P. J. Sanz, P. Ridao, G. Oliver, G. Casalino, C. Insaurralde, C. Silvestre, C. Melchiorri, A.Turetta: “TRIDENT: Recent Improvements about Autonomous Underwater Intervention Missions”. In IFAC Workshop on Navigation, Guidance and Control of Underwater Vehicles (NGCUV 2012), Porto (Portugal), April 2012. [link]