System Implementation

Subsystems

Localization

Our localization is based on a combination of six TOF (Time of Flight) sensors. We use the distances to the guardrail for generating position and orientation of the robot.

Our initial design for localization takes advantage of all six sensors and uses particle filter to get all-time localization. However, due to technical difficulties we reduced the scope to using 3 of the TOF's with the help of two limit switches to localize our robot relative to the guide rail.

The detailed working of localization procedures and state keeping documentation are in our github repository under jr-ros/jr_actuation

Localization algorithm running in V-REP simulation environment.

Particle filter using only 1 TOF sensor inside a closed rectangular space

Localization result 1 (using physical prototype)

Localization result 1 on prototype chassis

Localization result 2 (using simulation)

Locomotion

Thanks to our mecanum wheels configuration, our chassis is nonholonomic. To experiment with mecanum drive, we implemented the system in a simulation environment and tested the conversion from vx, vy, vw to the four wheel velocities. We also interfaced a joystick to the simulation so we can easily verify locomotion code correctness.

Once we had the completed physical chassis we tested the same pipeline on physical chassis and it worked perfectly. We are able to command the robot's state with a geometry_msgs/Twist where vx, vy and vw are all valid inputs at the same time.

For locomotion strategy on the test bed, we move along guide rails with help of TOF's and occasionally bump into the guide rail for state calibration.

Prototype chassis locomotion test

Robot locomotion in simulation with joystick control

Chassis

The chassis subsystem consists of a 8020 basic structure, four mecanum wheels, four DC motors and an aluminum plate that houses 6 ToF sensors, electronics box and a HEBI module arm assembly. The four mecanum wheels, along with four powerful DC motors, enable the robot to move in different directions carring the load. 8020 sructure adds extra reinforcemen to the main aluminum plate.

Chassis CAD model

Fully assembled 8020 structure

First chassis prototype

Mecanum wheel on 8020

Adding wheels to 8020

Fully assembled chassis with 8020 sturecture, wheels and aluminum plate

Computer Vision

The computer vision subsystem works on recognizing the device orientation and position with respect to the robot. The Intel RealSense captures the point cloud and figures out how far away the device is from the camera frame. In addition, the RealSense performs image segmentation on a 2D image of the device to recognize the orientation. The RealSense would recognize whether the intended device was positioned horizontally or vertically, as well as the displacement of the object with respect to the camera and arm. We would then send this information to the arm which would go to the intended location, and the different strategies depending on if the device was horizontal or vertical. However we noticed that due to discrepancies with in the arm position and RealSense calibration, we would not get to the exact positions we wanted. Therefore to ensure that our end effector would properly come in contact with the device, we added a fish-eye lens webcam to our end effector.

We used the webcam to make minor corrections to the arm positions. The webcam used image segmentation to locate the device, and then we tried to center the device with respect to the camera image.

A coolness factor for the RealSense was that it recognized the state the device was currently in (such as open and closed for the shuttle cock) and it compared it with desired state in the mission file. If the states matched then the RealSense ROS node sent out a message that we do not need to move the arm and we can move on to the next station.

RealSense Placement on Chassis

RealSense Device View

Segmented Image View

Webcam Placement on End Effector

View from Webcam as it is Making Position Corrections

Arm

The arm subsystem consists of a five degree of freedom arm using HEBI module and a hybrid end-effector. The arm provides the robot enough flexibility to manipulate the end-effector on the electromechanical devices. The end-effector has a granular jammer part and a solid beam part to operate different devices effectively.

Fully Assembled Arm Subsystem

On the left, is the fully assembled arm with the hybrid end effector moving to change the state of the vertical valve.

We use MoveIt's Trac-IK to solve the inverse kinematics for the Arm and HEBI API for the trajectory generation to plan the path from homing position to the target pose detected by the Realsense.

Partially assembled HEBI module arm

Materials for granular jammer

Testing granular jammer prototype

Onboard Electronics

We have a box (E-Box) housing all the onboard electronics. The E-Box contains power subsystem, compute subsystem, motor controllers and an emergency kill switch.

There are three batteries onboard: one 4S LiPo battery; one 36v LiGo battery and one 5v power bank. The LiPo battery powers the chassis motors; the 36v LiGo battery powers the Hebi arm modules and the 5v power bank powers all low voltage electronics including the NanoPi, the STM32 nucleo board and the motor controller boards.

For a topological map of electrical connections, refer to: System Design