Implementation

The Hardware

In order to have the right chords actuated for Baxter as he strums, we created a custom, secondary robot that mounts onto an ukulele. We wanted a device that could actuate quickly, play any chord, and given budget constraints, was as small as possible. Our design uses four subsystems (one for each string) consisting of a solenoid and motor resulting in eight actuators. Each solenoid has a 3D-printed "foot" that is toleranced to allow it to hold down it's respective string at the desired fret when actuated. Additionally, each solenoid is held within a carriage mounted to a lead screw such that, when it's respective motor is actuated, it can move to the desired fret position. The result of this is the system being able to play any ukulele chord when desired with the tolerancing such that each solenoid carriage can move past one another eliminating the need to make adjustments when switching chords. We use an ESP32 and I/O Port Expander to drive the chord playing mechanical system.

Additionally, to detect when Baxter has performed a strum, we take input from a sound sensor. This is also run through an audio analyzer which allows us to detect only frequencies produced by the ukulele and reduce noise from outside sounds. The input from the audio sensor reaches a threshold which triggers chord transitioning in the motors. Once the correct chord position based off each motor encoder value, the solenoids will actuate to play the chord. When the solenoids actuate, this triggers communication from the ESP32 and Port Expander, to an Arduino Uno. The Arduino Uno then publishes to Baxter to strum.

The Software

The UkƐ was designed using state space-based programing. It is initially off and idling. Once turned on, it is "paused" state where it waits for the play button to press before playing. When the play button is pressed, the solenoids transition to the next, pre-programmed, desired chord ("transitioning") and actuate. Once actuated, the device is in the "chord playing" state. When it first enters this state, the ESP32 microcontroller running the device communicates with an Arduino Uno . Using ROS serial the Arduino has already setup a publisher node and waits for the signal from the ESP32 before publishing a UInt16 message to the "strum_time" topic. Another node, running on the Baxter ROS master subscribes to this topic. When it reaches the correct value, it tells Baxter to strum by moving his joints to the correct positions. When Baxter has completed his strum, we will detect it with the audio sensor move and actuate the solenoids at the next chord position. The process will then repeat as long we want Baxter to play.

It was really important that we did not have Baxter strum the ukulele mid-chord transition. Not only would this sound bad, but the vibrations of the strings would cause the tolerances of the solenoid feet to be off and not centered on the strings. As such, it is essential that Baxter only strums once for each chord. To do this, we only publish the appropriate value to "strum_time" once for each chord. Additionally, after we have detected Baxter has strummed, we stop publishing the appropriate value, ensuring Baxter won't strum.

The Process

The final result of this design was a device that can play any chord in any sequence. And, with some simple code changes, any strum pattern. As such, Baxter can play any ukulele song he likes inspiring joy and amusement in his audience.

Page updated

Report abuse