Conclusion

Did We Meet Our Design Criteria?

Identify the location of each ingredient cup in its dexterous workspace.
- We can locate any cup within the dexterous workspace using AR tag tracking.
Pick up ingredient cups from arbitrary initial positions and pour their ingredients into the product cup.
- Since we can track where the cups are, using AR tags, we can move the Sawyer robot arm to that location, pick up the cup, and pour it using our Kinematics can Computer Vision related Nodes.
Utilize a scale to precisely know when to stop pouring the ingredient.
- We have successfully connected our Arduino and Scale setup to the rest of our ROS system and we can use the updates from the scale setup to tell our forward kinematics program to stop pouring.

Difficulties Faced

Camera Model URDF Replacing Sawyer URDF

When launching the intel realsense camera, it would also load a URDF model of the camera on the computer which would replace the Sawyer's URDF that would persist even after the computer shut down. ROS then tried to execute movement on the camera itself, instead of the robot. We had to find a launch file to restore the original URDF and changed to a camera launch file that did not load the URDF

Balls Bouncing out of Product Cup

Due to the curvature of the product cup and the rigidity of the balls, it was common during trial runs that the balls would bounce out of the product cup and onto the table. We found out that by wrapping the balls in gauze. While this prevented bouncing, it made it incredible hard to pour as it increased static friction. Instead we decided to line the bottom of the product cup with gauze to dampen the bounce, which was very effective.

Fragile Grippers

The slot design led to the mounting point of these grippers to be rather fragile. Regardless due to the nature of using a thin PLA 3D Print, they aren't the most durable. It was good enough for numerous test runs until we got our device working. Ultimately in a commercial product we would CNC or form a gripper out of a more durable material for early prototypes then likely injection mold during mass production.

Orientation Constraints

During the early stages of path planning, using the MoveIt library led us to using the built in Orientation Constraints. These built in constraints were very unreliable and often led to random movement after hours of troubleshooting. The workaround we arrived at was to cycle through 50 paths and choose the optimal shortest path that also maintained orientation due to the lack of excess motion.

Project Analysis

What potential social, political, and economic impacts could your project make?

Flow Bot can have many social and economic impacts depending on how it is used by the public. For instance, if Flow Bot is advanced to where it can successfully create mixed drinks with precision and accuracy every time then it can definitely be used in place of a bartender, cook, barista, etc. As farfetched as it sounds, robots are becoming more and more advanced as the years go by so it is perfectly plausible to believe that one day a robot may take your job. This can have a positive or negative impact on society, for one it could result in people losing their jobs to Flow Bot, which would affect them and their families emotionally and financially. On the other hand, Flow Bot could help business owners save money since they no longer need to pay employees. There are both winners and losers in this case. This is just one scenario where Flow Bot could have both a huge positive and negative impact on society. Flow Bot's ability of potentially take jobs away from people will eventually cause some type of political rift. The public will start arguing whether or not we should allow robots to do our jobs. Many people would claim it is morally wrong and unjust for their jobs to be taken away like this, while various business owners would likely argue the opposite since it benefits their business. If things escalate to where every day more and more robots are taking jobs then people will eventually riot and the political impact will definitely be grand. Of course, this is all just a prediction of what could happen, but given how technology continues to rule the world, these are all possible impacts Flow Bot could have on our society, economy, and politics.

Future Improvements

Machine learning for computer vision - Instead of using AR tags to detect objects, we could instead use a deep neural network like YOLO with the realsense depth camera on our RGB image to recognize objects and then map them to the point cloud from the depth camera. This was our original idea, but we did not have enough time to build a custom dataset and train the model. This would
Better controller - Controller currently still has robot arm shaking and will on occasion misalign with cups. Continuing to tune PID or changing to a different controller type like LQR or LQG could yield better results.
Dynamic Algorithm - Ability to dynamically change our jitter / pour based off of the material in the cup and its viscosity . Currently if we change to another material like beads instead of balls, we would have to change the pour / jitter angle and weight to pour on the scale. This would also require some sort of material detection algorithm (most likely some form of CNN).
Working Orientation Constraints - Exploring alternate Sawyer libraries with more accurate and responsive orientation constraints than those in MoveIt.
Moving Average for AR Tags - Currently the computer vision system uses the first value it read in for the location of the AR tag, which is very prone to Gaussian noise. To correct this we could use a moving average filter that averages the last n values to account for this noise rather than just using the first value read.
Better path planner / library - Our current solution of computing 50 paths and choosing the shortest one takes far too much time to compute and is not realistic for real world application. To fix this we should change to a library that solves for a shorter path more efficiently like PyKDL.

Page updated

Report abuse