Implementation

How It Works

Flowbot's system workflow is as follows:

Image Capture
- The Intel Realsense camera takes an image capture of the scene, once it recognizes all needed AR tags in the frame it moves forward to identify their locations.
Coordinate Frame Identification
- The coordinate frames of each AR tag are recorded and converted to points with respect to the Sawyer base frame. This is the same base frame used for its appendage actuation. This makes all actions the sawyer can make and the relevant object locations available in the same base frame.
Sawyer Pick-up Sequence
- Sawyer then runs through a sequence of motions using MoveIt to ensure it avoids all objects and smoothly picks up an ingredient cup. It is as follows:
  1. Sawyer tucks or returns to a set condition above the workstation
  2. Sawyer moves to a position directly above the cup offset by about 6 inches
  3. Sawyer moves directly down to a position where the grippers surround the cup
  4. The gripper closes and grasps the cup
Pouring Sequence
- Sawyer then runs through another sequence using MoveIt and joint manipulation to pour the ingredients to the desired amount into the product cup:
  1. Sawyer tucks or returns to a set condition above the workstation while carrying the cup
  2. Sawyer moves to an offset position behind the product cup
  3. Sawyer manipulates the J5 joint (See Pouring in Design) to pitch the cup in a jittering sequence slowly increasing in angle until contents begin to pour into the product cup
  4. Once the product cup reaches the desired mass identified by the scale module, the pouring sequence stops.
  5. Sawyer tucks once again with cup in hand
Sawyer Placing Sequence
- Finally, Sawyer moves the ingredient cup back to its original location and releases its grasp.
- It then either concludes the process or begins at Step 3 again with a new ingredient cup

Hardware Used

Sawyer Robot

The Sawyer Robot was the central piece of hardware for our system. We utilize its 6 DOF robotic arm to perform the actions of picking, placing, and pouring our cups.

Robotic Schematic of Sawyer

Gripper Module

The final iteration of the gripper functioned mostly as planned. The biggest downside was the number of prints needed due to grippers breaking during testing. It required 3 reprints and after it was calibrated correctly there was less concern for a need for another print.

Due to a lack of friction on the smooth PLA print, the use of rubber bands was helpful for added friction when grabbing the cup. This helped make sure it was secure and would wobble less during transfer.

Intel Realsense D435i Depth Camera

The camera was used to identify AR tags and provide 3D coordinates. This helped to identify the frames of the product cups, ingredient cups, and sawyer base. After transforming from the camera frame to the sawyer frame, the product and ingredient cup frames are then defined with respect to the sawyer base frame. This results in useful coordinates Sawyer can use to locate cups.

Scale Module

A scale was used to measure the amount poured into the product cup. A strain gauge was used to measure minute deflections as changes in resistance which was converted to mass values. These mass values were then processed and calibrated on an Arduino. This Arduino then published the data to our server and issued commands to tell our system when it was time to stop pouring after meeting the desired weight

Software

Sawyer Server Node (sawyer_server.py)
- This node, found in the planning package, is a server that waits for a message called enviro in the planning package. It's contents contain a geometry pose for the location and orientation of the desired cup sawyer is to move towards, as well as a list of values for a list of obstacles the client wants to tell sawyer to avoid. Once a request is sent to the Sawyer Server Node, it will then use the MoveIt! Path Planner and iterate through 50 different planned paths and find the path with shortest distance and move towards that location using the MoveIt! library to pick up the cup. We choose the shortest path as it happens to be the most direct and is less likely to knock over other cups with an extraneous path. In order to pick up our cups in their given locations, our process requires adequate accuracy in order to not miss or crush the cup with out grippers. When carrying out these transformations, we have implemented a PID controller to reduce any possible noise affecting our movement and thus reducing the size of our error to have precise actions.
- Files Used:
  - path_test.py
    - Once Sawyer Server Node receives an enviro msg, it utilizes path_test.py. Path_test calls path_planner.py in 50 iterations to find and keep the path_planning object with lowest amount of waypoints(shortest distance) and then calls controller to execute this plan with PID control
  - path_planner.py
    - Uses MoveIt! Library to generate path_planning objects to be used as directions for the sawyer arm to follow
  - controller.py
    - Uses a path_planning object along with preset PID control values to execute the planned path with closed loop feedback control to gain adequate accuracy
AR Tag Client Node
- Acts as main the computational and planning node in system by processing sensing information (scale weight, positions of ar_tags, and current Sawyer robot joint angles) and sending the correct positions and values to the inverse kinematic path planner node and the forward kinematics node.
- Files Used:
  - ar_tag_client.py
    - Finds locations of cups by calling ar_tag_controller.py and then converts the location of the cups from the camera frame to the base frame of the robot using Euler Angles and an offset. Finds current sawyer joint angles reading by calling robot_pos_sub.py and current scale value from scale_sub. The file then sends the desired position of the end effector to the Sawyer Server node over sawyer_parms/enviro to move the gripper around the dexterous workspace. To perform forward kinematics for the pouring and jitter motion, the node first sends a request to forward_kinematics_server over /forward_kinematics_postions to move to the pour position and then runs a while looping to keep moving the ingredient cup up and then down (jittering) until the ingredient cup has pour enough weight into the product cup. For every iteration of this while loop, the jitter will become larger until enough has been poured. The node then sends the desired position of the end effector to the Sawyer Server node to place the cup back down.
  - ar_tag_controller.py
    - Creates a tf buffer to lookup the transform between the camera and a given AR tag and returns the transformation.
  - robot_pos_sub.py
    - Acts as a subscriber to the robot/joint_states topic to get the current angles of the joints on the Sawyer robot. This allows us to perform forward kinematics relative to the current joint angles, so we don't move more joints than necessary.
  - scale_sub.py
    - Subscribes to the arduino/scalePour topic to determine how much weight has currently been poured onto the scale.
Forward Kinematics Server Node (forward_kinematics_server.py)
- Performs two types of forward kinematics on the Sawyer Robot. The first is moving the cup into a pour position by moving joint5. The second is jittering by moving between the pour position and to above the pour position by a given offset.
- forward_kinematics.srv
  - In the request of this service type there is a boolean called jitter, where is true the server will perform the jitter action, and if false will move joint5 to pour position, where the Sawyer Robot will rest in between jitters. The float32 named offset tells the node how large the jitter should be, as it gets progessively larger the longer the node is jittering. There is no response for this service type.
Arduino Scale Node
- Arduino_pub.ino
  - Uses the strain gauge library to calibrate and collect accurate mass measurements up to 5 kg. It then creates a publisher node and publishes these mass values through rosserial on the topic arduino/scalePour.
- Calibration.ino
  - Used to calibrate scale.

Page updated

Report abuse