Results & Conclusion

Overall the results of our project turned out better than expected, especially with the cross hatching drawings. We were able to recreate a variety of different styles of images and have the subject of the images be recognizable. We were also able to go beyond our initial goals of drawing with pointillism and managed to achieve fully automated multicolor cross hatching.

There were a significant number of challenges and difficulties that we faced that, and we weren't able to come up with ideal solutions for all of them.

The control aspect of the Sawyer arm did not have as much precision as expected, and this has led to a number of issues. We found that we could only manipulate joint angles to be within 0.005 radians of the desired angle, but this could potentially produce errors at least half a centimeter away from the desired position. In order to mitigate this, we tried decreasing the velocity of the joints, but this didn't appear to help. However, due to the nature of cross hatching not producing perfectly detailed and accurate drawings, this noise appeared acceptable in the resulting drawings.

The length of time it takes to draw pictures is quite a bit longer than expected. This issue was more pronounced when we were initially doing pointillism, but it ended up being somewhat mitigated through cross hatching. We tried adjusting various parameters to speed up the drawing, such as rearranging the order of the cross hatching lines being drawn, which appeared to be quite beneficial, and speeding up the joint velocity, which caused a degrade in the quality of the drawing and wasn't ideal. Because of the inertia of the arm itself, the pen would overshoot the end of the line at the higher joint speeds and draw longer lines than intended. One improvement that we did find was to apply a Gaussian blur before generating the cross hatching lines, which reduced the number of lines and also mostly preserved the quality of the resulting image.

Drawing straight lines was always a challenge especially with our control aspect only dictating the position of the end effector. We tried to draw lines as straight as possible by adding multiple waypoints along the line for the end effector to reach in order to correct for deviations, but due to the lack of precision with Sawyer, it wasn't able to straighten the lines much. With too many waypoints (e.g. one every mm or two), we found the lines to be very jagged, so we opted to go for fewer waypoints to create a more artistic feel with smoother but wavy lines. Again, this noise in the lines appeared acceptable in the resulting drawings.

Height calibration was the only manual process we had to do whenever we wanted to draw anything. Ideally, this was something we could automate into our code along with the canvas localization, but this remained an unresolved issue throughout our project. We were unable to use AR tags to determine the height of the canvas due to the substantial error margin along with the variety of different length pens. One option we didn't explore were integrating some pressure sensor into the end effector, but this was infeasible to incorporate into our end effector design and have connected to the rest of our setup. Another option was to use an external camera to visibly detect whether the tip of the pen touched the canvas, but this computer vision approach would require significant engineering effort beyond our primary project scope.

Although most of our resulting drawings as shown above resembled the source images quite closely, there were limitations on which kinds of images we could draw. The main limitation was that small shapes and details could not be accurately captured in our line drawings, since the primary purpose of cross hatching is to create the proper color or shading of a region. In order to remedy this, we tried determining edges of the image through computing the gradient or using Canny edge detection, but the effects of these edge lines appeared to only give minimal improvement in the quality of the resulting image compared to the number of additional lines drawn, which would significantly increase the time taken to draw the image.

For the multi-color end effector, there were significant delays in the manufacturing process due to long queues for the 3d printer. As a result, some parts of the end-effector were not printed. In particular, the revised versions of the pen clamps with additional clearance were not completed, and pen clamps from the first iteration had to be modified (manually) to fit the end-effector. As a result of this change, other parts also had to be modified, which included manual sanding/filing of rod clamps, drilling and punching of wooden offsets, and more. The modifications were successful and the design is fully functional, though the performance of the end-effector would have been better with more traditional manufacturing techniques such as milling and turning.

We chose to use the subtractive colors cyan, magenta, yellow, and black (CMYK) because of the behavior of light when adding pen ink. However, our sketch of The Starry Night turned out very dark.

We hope the artworks will inspire creative souls to enter the field of robotics, and that the public perception of the area is changed for the better.