Double Transducer: Force to Sound

A person presses down on something, which changes the color coming out of an LED, making it change color (from red to blue). This is read by a color reader, or sensor, that tells you what color the light is. The color change then causes the the sound coming out of a speaker, or sound making device, to change it's sound. As a person presses harder, the color changes from blue to red, which then changes the sound you hear.

This project definitely had some unforseen challenges for us, but we were happy to have a (mostly) functional final project on the day of critique. During our initial ideation we were drawn to using a color sensor in our system, but this turned out to be our biggest challenge. For starters, both the APD 9960 color sensor and the Neopixel RGB LED we were using required soldering to work, which made it difficult to get started right away. Once we were able to get past this first hurdle, the remaining electrical wiring was fairly straightforward, and we moved onto software, which required a lot of problem solving.

The hardest part about the code is attempting to let the RGB Neopixel display a rainbow range. Since the red, green, and blue values do not work as a single numeric value, it was difficult to translate the input from the FSR to a color value. We attempted to use nested "if" statements to modify different RGB values using segments of the input value. Our coding attempt is at the end of our code! In the end, we decided to vary the shade between red and blue: as blue goes down, red would go up. At zero force, the light would display blue. Then, as we increase the force, it would transition to purple -- eventually reaching red when the force is at the maximum.

Once we came up with a viable solution to that problem, things were relatively smooth sailing. The buzzer was fairly easy to code and get working, as was the tactile button, and we were able to use a prebuilt library for the color sensor. The final challenge we had to work through was getting the code for the LCD screen working and displaying the right values. While writing the code was fairly straight forward, we ran into an issue with the mapping. Since we were using a color sensor, the passage of data wasn't perfect, and we all ended up with different values on our displays. Initially we had been mapping it using 0-255 as the starting value, but in reality most of our color sensors never delivered anything over 170-200, so we ended up solving this by determining the highest value for each of our sensors, and mapping that instead.

Overall, going into the project we never expected these obstacles to be the main problems halting the development. The hardware wiring was fairly easy, but it was the seamless integration of multiple pieces and connecting this with software that proved challenging. Building specific sections of the project 1 by 1 is definitely a successful strategy to ensure no bugs and faulty sensors. Additionally, we found that creating clean code was extremely important, allowing each member to understand recent changes through minimal resistance. Each challenge provided an opportunity for each of us to takeaway an important lesson. These lessons provide meaningful insights that we can bring to the future projects in this class, as well as the building lifecycle in any future endeavors.

/*

  Project 1: Double Transducer

  60-223 Intro to Physical Computing

  Description: In this double transducer, we would use a force sensitive resistor

  to affect the color of an RGB LED. The color would then be interpreted by an Adafruit

  color sensor, and that color information would then be used to affect the frequency of a

  passive buzzer. The system would also include a tactile button which would allow the

  input from the force sensitive resistor to directly control the passive buzzer.

  Pin mapping:

  Arduino pin | role  |  description

  -------------------------------------

  A0            input    force sensitive resistor

  SCL + SDA     input    color sensor

  12            input    pushbutton

  A2            output   neoPixel

  4            output   small speaker

  Code by Kenneth Yu, Lenora Gant, Shivum Pandove

  Referencing the adafruit neopixel + color sensor library and Zach's LCD Screen code

*/

#include <PololuLedStrip.h>

#include "Adafruit_APDS9960.h"

#include <Wire.h>

#include <LiquidCrystal_I2C.h>         //library for LCD Screen

LiquidCrystal_I2C screen(0x27, 16, 2);  // set the LCD address to 0x27 for a 16 chars and 2 line display

Adafruit_APDS9960 apds;

//force sensitive resistor

const int fsrPin = A0;

//button

const int buttonPin = 12;

//neoPixel

const int neoPixelPin = A2;

const int numLed = 1;

//speaker

const int speakerPin = 4;

//variables for the screen

unsigned long printTime;

unsigned long trueTime;

//reading values from the sensors

int fsrVal, ledVal, colorSensorVal, buttonVal;

//mapped values

int mappedfsrVal, mappedColorSensorVal;

//output values

int rgbColorVal, speakerVal;

//for the screen

int LCDinput, MiddleOne, MiddleTwo, LCDoutput;

//light values for the rgb neopixel

int mappedRedValue, mappedBlueValue;

int mappedRedSensor, mappedGreenSensor, mappedBlueSensor, mappedClearSensor;

//LED output colors

int red;

int blue;

int green;

//LED sensor colors

uint16_t redSensor, greenSensor, blueSensor, clearSensor;

PololuLedStrip<neoPixelPin> ledStrip;

rgb_color colors[numLed];

void setup() {

  pinMode(fsrPin, INPUT);

  pinMode(neoPixelPin, OUTPUT);

  pinMode(buttonPin, INPUT_PULLUP);

  //initialize LCD

  screen.display();

  screen.init();

  screen.backlight();

  screen.home();

  Wire.begin();

  //start serial connection

  Serial.begin(9600);

  if (!apds.begin()) {

    Serial.println("failed to initialize device! Please check your wiring.");

  } else Serial.println("Device initialized!");

  apds.enableColor(true);

}

void loop() {

  // step 1: read all of the sensors

  readInputs();

  // step 2: make decisions about what to do

  updateInternalState();

  // step 3: drive all of the outputs

  driveOutputs();

  // step 4: report data back to the user

  reportBack();

  // a brief delay at the bottom of the loop is usually a good idea

  delay(5);

}

void readInputs() {

  buttonVal = digitalRead(buttonPin);

  fsrVal = analogRead(fsrPin);

  ledVal = analogRead(neoPixelPin);

  while (!apds.colorDataReady()) {

    delay(5);

  }

  apds.getColorData(&redSensor, &greenSensor, &blueSensor, &clearSensor);

  mappedRedSensor = map(redSensor, 30, 1300, 0, 99);

  mappedBlueSensor = map(blueSensor, 30, 1300, 0, 99);

}

void updateInternalState() {

  //mapping force value to 0-99

  mappedfsrVal = map(fsrVal, 0, 1023, 0, 99);

  //mapping force value into LED color value

  rgbColorVal = map(mappedfsrVal, 0, 99, 0, 255);

  red = rgbColorVal;

  blue = 255 - rgbColorVal;

  mappedRedValue = mappedfsrVal;

  mappedBlueValue = map(255 - rgbColorVal, 0, 255, 0, 99);

  green = 0;

  colors[0] = rgb_color(red, green, blue);

  if (buttonVal == LOW) {

    //If button is pressed, skip LED information

    speakerVal = map(fsrVal, 0, 1023, 20, 8000);

  } else {

    //if button is not pressed, map color information

    speakerVal = map(redSensor, 30,1300, 20, 8000);

  }

}

void driveOutputs() {

  if (buttonVal == HIGH) {

    ledStrip.write(colors, numLed);

  }

  //change speaker tone

  tone(speakerPin, speakerVal);

  delay(10);

  noTone(4);

  //mapping for the LCD Screen

  LCDinput = map(fsrVal, 0, 1023, 0, 99);

  MiddleOne = map(rgbColorVal, 0, 255, 0, 99);

  MiddleTwo = mappedRedSensor;

  LCDoutput = map(speakerVal, 20, 8000, 0, 99);

  //display

  if (millis() - printTime >= 250) {

    screen.clear();

    screen.setCursor(0,0);

    screen.print(String("i:") + String(LCDinput));

    screen.setCursor(6, 0);

    screen.print(String("m:") + String(MiddleOne));

    screen.setCursor(8,1);

    screen.print(String(MiddleTwo));

    screen.setCursor(12,0);

    screen.print(String("o:") + (LCDoutput));

    printTime = millis();

  }

}

void reportBack() {

  //prints for debugging

  Serial.println("Analog read = ");

  Serial.println(fsrVal);

  Serial.println("Red = ");

  Serial.println(mappedRedValue);

  Serial.println("Blue = ");

  Serial.println(mappedBlueValue);

  Serial.println("Red Sensor: ");

  Serial.println(mappedRedSensor);

  Serial.println("Blue Sensor: ");

  Serial.println(mappedBlueSensor);

}

/* scrapped code to cover the whole rainbow (failed pretty badly)

  if (rgbColorVal > 256){

    red = 255;

    blue = rgbColorVal;

    green = 0;

  }

  else if (rgbColorVal < 512){

    red = 511- rgbColorVal;

    blue = 255;

    green = 0;

  }

  else if (rgbColorVal < 768){

    red = 0;

    blue = 255;

    green = rgbColorVal - 512;

  }

  else if (rgbColorVal < 1024){

    red = 0;

    blue = 1023 - rgbColorVal;

    green = 255;

  }

  else if (rgbColorVal < 1280){

    red = rgbColorVal - 1024;

    blue = 0;

    green = 255;

  }

  else if (rgbColorVal >= 1280){

    red = 255;

    blue = 0;

    green = 1535 - rgbColorVal;

  }

*/