Our goal is to get Baxter to be able to play the ukulele. We envision him, with one hand, holding a custom built self-actuating ukulele that can play pre-specified chords, and then, with the other hand, be able to strum the ukulele at a given rate to generate music. The custom built ukulele would be built in such a way that whenever a chord is held, the device will use ROS and publish a set of motions mimicking strumming to Baxter. Cameras will be integrated to allow for better accuracy when executing the strumming on the instrument. Once the strumming action has been completed (as sensed with vibration sensors), more information would be published to command the ukulele to actuate to the next chord. Additionally, dancing motions and other actions could be programmed into Baxter’s movement to enhance his performance.

This is an interesting project as it demonstrates how robots can perform complex, precise and delicate tasks in the same manner as a human. A successful implementation of ukulele playing will give us confidence that we can control robots to perform many other unique but equally complex actions. While Baxter might not strum and play a song at the same speed as a human right now, maybe one day it shall be possible.

This project also involves the integration of several different hardware and software components. We had the opportunity to learn about systems beyond just Baxter and ROS, and how to create a complete design in which the systems can interface with one another.

With robots that can operate instruments, we can enable composers to play the music they write without having to perform the motions themselves. This can allow more complex scores to be written, or give amputees the ability to continue playing music. We could even have entire performances by robots, which would make musical performances more accessible to a wider audience.

The ability to perform precise motions can also be applied to fields beyond music. For example, medical research involves microscopic tasks that are limited by the precision and throughout of human scientists. While having Baxter strum a ukulele is a far stretch from cellular pick and place tasks, the same ideas and implementations can be improved upon in order to achieve this.