During the summer of 2019 I had the opportunity to work as an undergraduate research assistant at the University of Manitoba's Fluid Power and Telerobotics Lab. While there I conducted research on integrating computer vision into a teleoperated hydraulic arm
Using OpenCV and ArUco markers I replaced the hand-held manipulator teleoperation controls of the arm with computer vision controls that mapped the movement of the fiducial marker to the movements of the arms end-effector.
Simultaneous videos at the same moment in time from both viewing the robot and the cameras perspective:
Utilizing ArUco markers to detect when a worker enters into a robot's work envelope, in real-time the work envelope of the robot is limited by applying force feedback into the hand-held manipulator, preventing the robot from endangering the worker.