Embedded Files
Aarush Das, Kireeti Chilakamarry, Sarvagya Somvanshi, John Chang
Abstract
Abstract
Our goal with this project is to replicate scrolling motion (colloquially known as "doomscrolling") using a UR7e robot. Scrolling is an extremely challenging motion to replicate, especially with finicky hardware, as it requires us to execute each step of the motion with precision. Otherwise, slight variances at each step of the way can cascade downstream and potentially prevent the robot from successfully scrolling. This is especially important in our case, since doomscrolling repeats this scrolling pattern indefinitely.
The methods used to combat these variance-related issues can be applied to other repetitive pick-and-place motions that require the utmost precision, such as warehouse robots or any other arm that requires this form of motion.
Our project uses a camera that has view of 3 AruCo tags: base of the arm, stylus location, iPad location. The robot then picks up the stylus from a 3D printed holder, followed by a motion to move over the iPad, and then finally move down to the iPad to scroll on a social media platform for any given amount of scrolls set by the user.
Project Challenges
Project Challenges
Our project can be divided into two main challenges: mitigating variability and accurately detecting AruCo tags.
Mitigating Variability: The margin of error on the arm was too large for the tolerance range we needed to not break the iPad but still reach down low enough to touch the screen with the right amount of pressure. To address this, we started by using a stylus with a soft tip, which would enable us to have some leeway in the z-direction (up and down) on the iPad screen as opposed to using a hard tip pen, such as an Apple Pencil. During our demo, we also opted to add a human-adjustment feature that would reliably get us to the same pressure needed to scroll on the iPad 100% of the time. The way our human-adjustment feature works is at the moment before the arm reaches down to scroll, the arm pauses and waits for commands in the terminal to either go up or down a couple millimeters until the stylus started to touch the screen. This led to a successful demo and much more reliable results (without the human-adjustment feature, our success rate was roughly 60%).
Detecting AruCo Tags: What made this project very interesting to us was the computer vision aspect, where after seeing an image of where the 3 AruCo tags were setup, the arm has the ability to plan a motion through inverse kinematics to achieve the task of scrolling. However, to set this correctly, we experimented with numerous camera angles and AruCo tag sizes and poses to figure out which configuration would achieve the highest accuracy and precision. Initially, we started with all three tags being the same size, but then quickly adapted for smaller tags on the stylus and iPad to reduce the margin of error the camera may have in determining the center of the tag, which our offsets for the trajectory were based on. We also opted to have our camera placed at a bird's-eye view high above the table to reduce the distortion of the 2 small tags parallel to the table, while also being able to observe the angle of the AruCo tag on the base of the robot, so that we did not lose any dimensionality and have any singular configurations.
Page updated
Report abuse