Learning for intervention
Scenario definitions
Scenario 1: Grasping Object: black-box Approach: Top
Scenario 1: Grasping Object: black-box Approach: Lateral
Scenario 1: Grasping Object: Anphora Approach: Top
Scenario 1: Grasping Object: Anphora Approach: Lateral
Scenario 2: Pushing Object: black-box Approach: Lateral 1
Scenario 2: Pushing Object: black-box Approach: Lateral 2
Scenario 2: Pushing Object: Anphora Approach: Lateral 2
3D Reconstruction Experiments
The robot arm performs a linear movement (joint q2) to produce a laser sweep. A virtual camera captures the scene and a 3D reconstruction algorithm creates a 3D reconstruction of the scene (Point Cloud Data).
Generated Point Cloud: [output.pcd]
Teleoperation Experiments
Scenario configuration:
the robot is initially placed at a fixed position P1 (but can also be teleoperated in x,y,z)
an object (i.e. a black-blox) is laying on the floor and can interact dynamically with other objects (physics engine)
the arm can be teleoperated in joint space (q0, q1, q2, q3, q4, q5)
the user gets visual feedback from the simulator (the 3D scene + contact information)
contact information is received now in binary form (true/false) on the screen. True indicates that there is a contact between the robot and any other object (i.e. the black-box).
For the teleoperation, the user uses a gamepad.
During the teleoperation and training stage, the system stores all the information in a bagfile. Then, a Matlab file with the proper variables {Timestamp, q0, q1, q2, q3, q4, q5, Contact} is automatically generated for posterior processing.
Experiment 01
Variables: {Timestamp, q0, q1, q2, q3, q4, q5, Contact}
Matlab file: 001.m
Graphical representation
TRITON project
2 underwater validation scenarios:
(1) Black-box (flight recorder) grasping & recovery
(2) Panel intervention: connect / disconnect / turn a valve
website: http://www.irs.uji.es/triton/
IRSLab YouTube channel: http://www.youtube.com/user/IRSLab
Vehicle: Girona500
Ribas, D.; Palomeras, N.; Ridao, P.; Carreras, M.; Mallios, A., "Girona 500 AUV: From Survey to Intervention," Mechatronics, IEEE/ASME Transactions on , vol.17, no.1, pp.46,53, Feb. 2012 [doi]
Robotic arm: Lightweight ARM5E
4 D.O.F. + 1 Gripper
[Fernández12] J. J. Fernández, M. Prats, J. C. García, R. Marín, P.J. Sanz and A. Peñalver. “Manipulation in the Seabed: A New Underwater Robot Arm for Shallow Water Intervention”, 1st IFAC Conference on Embedded Systems, Computational Intelligence and Telematics in Control (CESCIT), pp. 314-319 ISBN: 978-3-902661-97-5, 3-5 April 2012, Würzburg, Germany. [on-line]
[Fernández13] J. J. Fernández, M. Prats, P. J. Sanz, J. C. García, R. Marin, M. Robinson, D. Ribas, P. Ridao, "Manipulation in the Seabed: A New Underwater Robot Arm for Shallow Water Intervention", in IEEE Robotics and Automation Magazine, to appear in 2013.
Underwater Simulator (UWSim)
Website: http://www.irs.uji.es/uwsim/
Prats, M.; Perez, J.; Fernandez, J.J.; Sanz, P.J., "An open source tool for simulation and supervision of underwater intervention missions", 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2577-2582, 7-12 Oct. 2012 [doi]
ROS interfaces
Simulated TRITON scenarios:
Scenario 1: Black-box (flight recorder) grasping & recovery
Scenario 2: Panel intervention
Sensors / system inputs
Arm joint values: Slew, Shoulder, Elbow, JawRotate (4 D.O.F.)
Arm gripper: JawOpening
End-effector cartesian position + orientation: (x,y,z,r,p,y)
JR3 90M31 force sensor [details]
Pan/tilt camera ECA-DTR100ZC [details]
Stereo Camera VIDERE STH-MDCS [details]
Tactile sensors Tekscan FlexiForce [details]
Tritech SeaStripe laser for 3D reconstruction [details]
Control action / system outputs
Arm joint values: Slew, Shoulder, Elbow, JawRotate (4 D.O.F.)
Arm gripper: JawOpening
End-effector cartesian position + orientation: (x,y,z,r,p,y)
User interfaces / feedback
Multimodal user interfaces. (Not yet implemented on UWSim)
[Garcia10] J. C. García, J. J. Fernández, P. J. Sanz, R. Marín, Increasing Autonomy within Underwater Intervention Scenarios: The user interface approach, 4th Annual IEEE International System Conference, pp. 71-75, San Diego (USA), April-2010. [doi]
[Marin02] R. Marín, P.J. Sanz, A.P. del Pobil. “Teleoperated Robot System Via Web: The UJI Telerobotic Training System”. International Journal of Robotics and Automation, Special Issue on Web-based Robotics and Automation. Volume 17/Number 3/2002, paper 105, ACTA Press, 2002.
On-screen feedback information (not implemented)
Gamepad (cartesian / articular spaces) (not configured on simulator)
SMI iView eye tracker [specs]
Diego R. Faria, Ricardo Martins, Jorge Lobo, Jorge Dias, Extracting data from human manipulation of objects towards improving autonomous robotic grasping, Robotics and Autonomous Systems, Volume 60, Issue 3, March 2012, Pages 396-410, ISSN 0921-8890. [online]
Procedures to learn
Robot end effector plug connector on the valve (following the apropiate orientation)
Robot end effector unplug connector from the valve
Robot end effector grasps the black-box in a stable manner
Learning techniques
Supervised learning (bayesian approach)
Learning by demonstration
Reinforcement learning (?)
Related references
[Prats13] M. Prats, Á. P. del Pobil, P. J. Sanz, “Robot Physical Interaction through the combination of Vision, Tactile and Force Feedback”, Springer Tracts in Advanced Robotics, Volume 84 2013. ISBN: 978-3-642-33240-1 (Print) 978-3-642-33241-8 (Online). http://dx.doi.org/10.1007/978-3-642-33241-8.
[Prats12a] M. Prats, D. Ribas, N. Palomeras, J. C. García, V. Nannen, S. Wirth, J. J. Fernández, J. P. Beltrán, R. Campos, P. Ridao, P. J. Sanz, G. Oliver, M. Carreras, N. Gracias, R. Marín, A. Ortiz. “Reconfigurable AUV for intervention missions: a case study on underwater object recovery”,. Intelligent Service Robotics, Volume 5 (1), pp. 19-31, January 2012. [link]
[Prats12b] M. Prats, J. J. Fernández, P. J. Sanz, “Combining Template Tracking and Laser Peak Detection for 3D Reconstruction and Grasping in Underwater Environments”, Proceedings of the 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 106-112, Vilamoura, Algarve (Portugal). 978-1-4673-1735-1/12/S31.00. , October 2012 [doi]
[Sanz12] P. J. Sanz, P. Ridao, G. Oliver, G. Casalino, C. Insaurralde, C. Silvestre, C. Melchiorri, A.Turetta: “TRIDENT: Recent Improvements about Autonomous Underwater Intervention Missions”. In IFAC Workshop on Navigation, Guidance and Control of Underwater Vehicles (NGCUV 2012), Porto (Portugal), April 2012. [link]