Embedded Files
Robotics has become increasingly accessibly to the average person in the past few years. Electronic parts used to assemble robotic manipulators are becoming increasingly cheap to manufacture, not only at a mass production level, but also for small-order custom parts e.g. for the aspiring hobbyist or avid prototype designer. Robotics software is being written in higher level languages e.g. Robot Operating System (ROS) has a Python API rospy. Continuing this trend of accessibility, we aimed to make user-directed robotic manipulation of the environment as straightforward as possible.
The idea is that the user begins in Gazebo and creates a custom and unique structure, then chooses a few camera angles that they think best represent their structure. Then, they run our project on the specified camera angles. Finally, the user sits back and watches the robot build their custom structure. Gazebo's drag-and-drop friendly interface enables users to quickly and easily create structures of their choosing. If a user wants a more standardized structure to begin with, we provide them with an API interface to e.g. add a square of blocks. In terms of robotics hardware accessibility, we chose TIAGo, a popular and open-source robot. TIAGo possesses the two primary qualities in robot that we were seeking: the ability to move in its environment in order to have the greatest spread of environmental effectiveness and a precise manipulator to enable a wider range of actions.
Our project, nicknamed "roBob" for its likeness to the children's show "Bob the Builder", involves six main steps.
Image a structure
2. Reconstruct a schematic of that structure leveraging those images
3. Plan a reasonable ordering to build the structure
4. Identify where the blocks are in the world
5. Navigate the environment until within grasping distance of a block
6. Grasp the block (and move and place the block in the correct location)
Our project is interesting because it involves the very challenging image processing component of determining the locations of each block in a structure based on images of the structure. This is difficult because we must recreate a 3d model based on just images, and after recreating the 3d model we have to separate the model into blocks and determine the position of each block. The planning of how to best pick up and place the block is also challenging, as we don’t want to drop the block and we don’t want to push other blocks in the structure when we place a block. Also, since TIAGo doesn’t originally know where the blocks are in its world, our project also involves exploring the world to find the blocks using SLAM.
Complexity breakdown by sensing, planning, and actuation
Complexity breakdown by sensing, planning, and actuation
Sensing
Taking images/processing images to determine position of blocks in structure
Uses laser data to determine roBob's position in world and position of blocks lying around roBob's world
Uses laser to avoid obstacles (blocks lying around) when moving around world
Planning
Planning for original loop around world to determine positions of blocks (where to explore next while avoiding obstacles)
Planning for how to move around world and go towards blocks/towards structure, while avoiding obstacles (using SLAM)
Planning for how to pick up/place blocks using MoveIt Controller (how to orient arm to pick up and place a block, when to close/open gripper)
Actuation
Move TIAGo to desired location by setting desired velocity every few timesteps
Move TIAGo’s joints to a position such that it can pick up/place a block
Open/close gripper
Page updated
Report abuse