Results

Summary

We were successfully able to run through the pipeline of imaging a structure, generating the corners, and having Tiago rebuild the structure using the blocks in its world. We were able to repeat this process with various structures, but we did have to make a few changes from our original design:

  1. Our robot is only able to rebuild the first layer of the structure, as we don't yet have a consistent way to put blocks on top of each other.

  2. The blocks in the rebuilt structure are more spaced out than they were in the original structure, so that Tiago doesn't hit the other blocks when placing down a block.

  3. We use a placeholder marker to tell Tiago where to place the next block, as gmapping isn't accurate enough to tell us Tiago's exact location in the world (and so given the same goal location, it might go to a slightly different place every time)

Results (CV)

We are very happy with how accurately and consistently we were able to determine the corners of the blocks in the structure. The videos below demonstrate the process of taking images of three different structures, and reconstructing the corners from these images. For each of our structures, we used the same predefined camera positions and angles (8 pairs of images total, taken by one camera). We also offset each structure differently to make sure our algorithm would still work when there wasn't a corner that intersected with the origin.

We also wanted to test how well our imaging process would work if we used fewer images. The graph below shows a test we conducted on one structure, where we looped through all possible numbers of pairs of images, randomly chose that number of pairs from the images we had, and tried to reconstruct the corners from just the chosen pairs of images. We repeated this 5 times for every number of pairs, and averaged the results. We found that we needed to use all 8 pairs of cameras to reconstruct all the corners, and that the number of pairs cameras directly correlated with the number of corners recovered.

Results (Robob)

As a whole, our robot fell a little short of the initial design ideas that we had. Although it was able to put together the structure, it was unable to do so consistently, and it was unable to stack blocks on top of each other or put blocks directly side by side. As a result, it was only able to successfully build structures involving blocks spaced apart. We ended up successfully picking up and placing blocks down around 70% of the time.

However, aside from placing blocks down, we were able to successfully identify blocks around the area simply by having the robot look around. It was also able to autonomously navigate to the blocks and the final structure location. Once in front of blocks, it was successfully able to identify the marker and create an expected pose for the block before picking it up. It was also able to identify markers representing where to place blocks, though it struggled with putting them down properly. Below are the sped-up and full videos of Tiago building a 5 block structure.