Embedded Files
Final Project Video
Final Project Video
Overall, Drawyer performed decently well! It was able to successfully filter out the background of an input image (1) to one with just an outline of user's face/body (2). Unfortunately, Drawyer had missed all the facial features in this trial (but had them in other trials) so we may need to work on the image processing and possibly implement facial recognition to get a more accurate outline of just a person's face with important features like the nose, mouth, eyes, etc.
However, Drawyer drew the outlined image almost perfectly: Sawyer's rendition (3) is virtually the same as the processed image (2)!
(1) Raw Input Image
(2) Processed Image
(3) Sawyer's Drawing
Conclusion
Conclusion
Drawyer did an excellent job meeting its original goals: it first took in camera input involving a person with some background, distinguished the face and its key features in the image, and converted it to a simple outlined image which Sawyer was able to accurately draw on paper. It met all of our basic design criteria.
As mentioned earlier, it did have some setbacks, such as not precisely retaining the face for different input images. This was because Canny edge detection on its own does not remove all unwanted noise from the image. We attempted using different filters (Bilateral, Gaussian), but found the best general results with median filtering. Each image needs to have noise/threshold parameters manually tuned, which is why we added the option for the user to change these parameters to their liking.
Additionally, we had some difficulty with the trajectory planning: we started by trying to use controls and planning a path using only the final pose with orientation constraints. However, Sawyer could not take the shortest path between points with a reasonable trajectory. Thus, we switched to using inverse kinematics using a simple IK solver which would give us the easiest path, which worked reasonably well.
In the Future
In the Future
Implement shading, coloring, and brush weight/angle: Using a more advanced end effector, we can utilize coloring and brush weight. With some slight modifications to Drawyer's code, shading as well as brush weight/angle could be achieved.
Designing a 3D printed, spring-loaded pen gripper would allow the pen to adapt to slight changes in the tabletop. (Our current system attaches a brush pen directly to the rigid standard Sawyer gripper.) This modification would likely yield smoother linework and consistent pressure/friction between the pen tip and page.
We could improve Drawyer motion planning to draw in steady strokes, rather than the 8, basic, pixel-length motions it currently uses. We attempted steady strokes from an initial point to an end point, but, because of Sawyer's tendency to take random paths, we opted for a simpler IK solver for small pixel-length motions.
AR tags could allow Drawyer to automatically calibrate it's end effector to a page of different orientations. This would free Drawyer from the restriction of drawing on a flat level surface as in this prototype.
Report abuse