Our project consists of three parts that work on concert to go from image to launch:
Target Acquisition: Take an image from the onboard camera, find the target, then determine the desired launcher position in 3-space
Robot Control: Take the desired launcher position, then move to the correct position and orientation
Custom Dart Launcher: After moving to the correct position, fire the dart to hit the target with a predictable trajectory
To enable our system to reliably launch the dart, we require an extremely reliable dart launcher, which would repeatably launch the dart on a predictable trajectory. To further constrain the design, additional design choices were made. For minimal control complexity, the design required interfacing with the built-in gripper mechanism, relying no new actuation for the Sawyer to trigger a dart launch. Finally, for increased flexibility, variable launch power was desired.
To meet these requirements, we designed and implemented a spring-powered dart launcher. The launcher uses a separate spring-assisted trigger arm to pull out the trigger pin with the existing gripper actuation. Additionally, multiple slots for the trigger pin allowed for multiple power settings for launcher. However, this simplicity of actuation comes at a cost: we require human intervention to reload the dart between launches. For a real-world application, the system would be far more robust if it had a self-loading actuator and mechanism as well.
Our target acquisition method requires a few steps to go from raw image to desired launcher location. First, we determine the target's x, y, and radius in the raw camera feed using the Hough Circle Transform. Because we know the size of our target, we establish a linear relation between target distance and radius, allowing us to get a reasonable target distance, which we can then use with the camera matrix to determine the spatial coordinate frame of the target.
Next, to get the desired position of the launcher, we use a combination of the kinematics of the dart combined with experimentally determined characterization of the launcher to position the launcher at the correct angle and height in order to hit the target. Finally, the resulting twist is sent to the move planner for actuation.
In order to accurately launch the dart, we require a control and path planning pipeline that is robust, consistent, and repeatable in order to place the robot end effector, and transitively the dart launcher, at the desired position and orientation. It also needs to avoid collisions with surrounding obstacles and have smooth movement to avoid jerkiness and vibration that may cause mechanical failures.
To meet these requirements, the MoveIt path planner, a PID controller, and inverse kinematics were used to control the robot movement. PP_move listens to the target acquisition module for the target pose and instantiates the PID controller and the MoveIt planner, which are used to plan and control the movement of the arm to the desired position while allowing for collision avoidance, short solution computation time, and relatively precise movement.