Implementation

Hardware

Baxter Robot

We used Baxter, a two-armed robot with seven joints on each arm.

Kinect cameras

We used Kinect cameras for computer vision because of their ability to both capture point-cloud data as well as 2-D images.

Software

The overall workflow of the our project can be broken down into 2 overall phases: grasping the board and ball bouncing (the ball bouncing phase is the more substantial of the two). We break down each part below along with their specific implementation details.

Board gripping procedure

MoveIT inverse kinematics and controller configuration

Many of the steps above rely on MoveIT for inverse kinematics solutions, as well as for using the PID controller. To improve the performance of these pieces, we make the following changes to the standard MoveIt setup:

We configured the IK solver to determine a solution which minimizes the condition number of the Jacobian Matrix; this results in a more stable solution, and hence more reliable movement.
We employ the TracIK solver, as it produces inverse kinematics solutions at a higher success rate.
We scaled down the max velocity and acceleration to ensure slower joint movements, and also reduced the goal deviation threshold so that the final joint positions were closer to the true target positions.

Ball Bouncing Procedure

Node 1: Ball Tracking + Centroid Calculation

Point Cloud Technique

Our Ball tracking + Stereo node subscribes to point cloud data from each of the Kinects and it projects the 3D point cloud onto a 2D image using the known camera lens properties. Since the only green-colored object in our world is the ball, we can use RGB filtering to filter pixels that do not correspond to our green ball. The node then finds the median of the surface points corresponding to our green ball to solve for the centroid of the ball. Finally, it projects the centroid of the ball in the 2D image onto the reference frame of the camera and applies a transformation to find the points in the world reference frame. We apply this procedure from three different Kinects in order to recover points from the entire surface of the ball. Once we have our centroid in our world reference frame we pass the points to our Kalman filter to reduce noise and make other important computations. Our point cloud technique accurately localized the centroid of the ball. However, it took longer to compute than our Stereo technique.

Stereo Technique

We also experimented with a stereo technique to localize the ball as it was falling. We first filtered for the green ball directly from a snapshot of a 2-D pixelated image taken from the Kinect camera. We then drew a bounding box using OpenCV and, from here, computed the centroid of the ball. We deduced that our computed centroid from each of the 2-D images corresponded to the same point in the 3-D coordinate frame, and consequently, used triangulation and linear least squares to calculate the depth and ultimately the world coordinates of the centroid of the ball. This technique proved to be faster but less accurate than our Point cloud approach that we decided to implement for our project.

Node 2: Kalman Filtering

In order to mitigate against any sensing noise from the ball tracking node, and to allow for tracking of the ball velocity, we developed a custom Kalman Filter which, given a simple dynamics model formulated using kinematics equations, outputs an updated centroid location, as well as the centroid's velocity. The Kalman filter proved essential, as it was able to robustly infer velocity information which was critical to computing the target board tilt, and also was robust to any deviations from expected movements, such as the ball bouncing off the board.

Node 3: Physics Inference

Given the ball's position and velocity, the physics inference node computes the direction and angle in which to tilt the board. This computational process can be further broken down into three steps:

Using the equations of projectile motion, compute the position and velocity at which the ball will strike the board.
Given this contact information, determine what the axis along which to tilt, and the target "exit velocity" of the ball should be in order to redirect its motion back towards the middle. For instance, if the ball is to the left of the center, the exit velocity would be pointing to the right, and its magnitude and exact direction would depend on the extent to which it is away from the middle. The determination of which axis to tilt along is based on which axis the ball is furthest from the board's center.
Compute the board tilt angle along the given axis to achieve the previously computed exit target velocity.

Node 4: Tilt Controller

The tilt controller allowed the robot to tilt the board along either the x-axis, or the y-axis. The control logic for each case was separated:

X-axis tilt

To tilt the board at an angle 𝜃 along the x-axis, we directly modify the Baxter robot's 1st elbow joint in an open loop manner. The main challenge with this, however, was that both arms needed to move simultaneously in a synchronized manner in order to robustly move to the proper target position. To achieve this, we computed the angle by which to move the elbow joint of the robot, and quantized this movement into smaller sub movements (the level of quantization was tuned). Each arm then alternated in moving by these small amounts, resulting in a net effect of the arms moving in a nearly synchronous manner.

Y-axis tilt

In order to tilt the board at an angle 𝜃 along the y-axis, we computed which height difference between the two end effector would achieve this tilt angle. One this height difference, h, was computed, we then move each arm up, or down (depending on the sign of 𝜃), by an amount of h/2. This movement is achieved by adjusting the target end effector z-coordinate accordingly, and by computing the joint angles which achieve this new target position, with an orientation constraint, using MoveIt's inverse kinematics solver. Given these target joint angles, we use Open-Loop control to move the robot to this position. However, to avoid jerky or sudden movements, we additionally implemented a low-pass filter on top of this open-loop controller; this strategy ensures that the movement to the goal joint angles will slowly traverse the line segment between the initial joint angle values and the goal joint angles. Each arm was moved separately (i.e. first the right arm moved, then the left).

After each movement, the board was returned to a pre-computed neutral position, to ensure that all controller computations could begin from the same starting position.

Page updated

Report abuse