#Final: Infinity Gauntlet

Video Documentation

CONCEPTION AND DESIGN

Our project explores the concept of supreme power through an interactive glove that symbolizes infinite abilities—the power to create or destroy. Using Arduino and Processing, we designed natural scenes where users interact with living creatures through hand movements. For instance, bending the elbow and lifting the hand causes tree to grow, pinching fingers creates little life like butterflies and birds, and punching forward generates thunder and lightning, striking what users have created. These visual and aural effects aim to immerse users in a powerful experience that inspires reflection on humanity’s relationship with power and responsibility. Drawing inspiration from Professor Marcela’s CINEKID FESTIVAL project, which emphasizes empowerment, we incorporated the concept of the Infinity Gauntlet to provoke contemplation of the ethical implications of wielding such power.

User testing revealed areas for improvement. Participants suggested providing a clearer introduction and demonstration to convey our concept effectively. Additionally, we found that the flex sensor in the gauntlet was prone to unreliable triggering, prompting us to redesign the interaction mechanism for greater accuracy.

One of our earliest challenges was selecting the right sensor to detect hand-lifting and punching motions. Initially, I tested an accelerometer, but its readings were affected by orientation, making it inaccurate. I then tried a muscle sensor with adhesive electrodes, but it was unsuitable for repeated use by multiple users. Another muscle sensor had insufficient precision, as its value changes were too small to reliably transfer useful signals. Ultimately, I switched to the IMU, which detects acceleration and gyroscope values, allowing me to use the Z-axis acceleration and Y-axis gyroscope for hand-lifting detection and the combined X-Y acceleration velocity for punching detection. This effectively minimized over-triggering and ensured accurate gesture recognition.

Another setback involved the use of Neopixels. We initially planned to integrate these for dynamic light effects to enhance the visual experience, such as bubbling stream or lightning strikes. However, incorporating Neopixel code caused significant delays in data transfer between Arduino and Processing, disrupting the performance of other components. Due to these technical limitations, we decided to omit this feature. 

The most significant challenge lay in debugging and structuring the complex logic of our code. Since multiple effects were triggered by overlapping gestures, integrating separate code sections revealed minor issues that were unnoticed individually but significantly disrupted functionality when combined, affecting the performance of other features. To address this, I broke the project into stages, drafted the logic on paper, and emphasized modularized coding with clear comments. This systematic approach not only improved functionality but also deepened our understanding of managing multi-layered interactions in a collaborative project.

Sketches

CONCLUSIONS

Our goal was to empower users and provoke reflection on the relationship between power, privilege, and responsibility. At the IMA show, users engaged enthusiastically with our project, often exclaiming in excitement when activating lightning and thunder by punching, and smiling in surprise when creating butterflies. One user remarked that wearing the glove and seeing movements translated into responses was a unique and powerful experience. Their reactions and  feedback exceeded our expectations and highlighted the immersive and meaningful interaction we aimed to achieve. When users learned to control their power, they explored creation and destruction intentionally, aligning perfectly with my definition of interaction as an ongoing, initiative-driven dialogue between the project and its audience.

Despite its success, the project has areas for improvement. First, users suggested enhancing the destructive power by resetting the tree’s growth upon detecting a punch. Second, incorporating distinct endings based on the user’s choice to create or destroy could better reinforce our concept. For example, a tree flourishing as a result of creation or withering after destruction would provide a clearer and more impactful conclusion. Third, the glove’s design should be refined to make it easier to wear and remove.

DISASSEMBLY

APPENDIX

Wiring Diagram

Arduino code

#include <Adafruit_MPU6050.h>

#include <Adafruit_Sensor.h>

#include <Wire.h>

Adafruit_MPU6050 mpu;

//MPU 6050 sensor

float ax, ay, az, gy;

const float thresholdPunch = 5.0;     // Threshold for XY acceleration magnitude (punch detection)

const float thresholdLiftGyroY = 1;   // Threshold for gyroscope Y (hand lifting)

const int sustainCountThreshold = 3;  // Number of consistent readings to confirm lift

float preXY = 0.0;                    // Previous XY magnitude

int liftCounter = 0;                  // Counter for sustained lifting motion

unsigned long previousTime = 0;

const unsigned long interval = 20;  // Sampling interval in ms

//force sensors

int forcePin1 = A0;

int forceValue1 = 0;

int threshold1 = 800;

int val;  // whether the values of the force sensor are above the thresholds

int preVal = LOW;

int count = 0;  // count the times of clenching

bool pinch = 0;

int punchDetected = 0;

void setup() {

 Serial.begin(115200);

 while (!Serial) {

   delay(10);

 }

 if (!mpu.begin()) {

   Serial.println("Failed to find MPU6050 chip");

   while (1) {

     delay(10);

   }

 }

 mpu.setAccelerometerRange(MPU6050_RANGE_16_G);

 mpu.setGyroRange(MPU6050_RANGE_250_DEG);

 mpu.setFilterBandwidth(MPU6050_BAND_21_HZ);

 Serial.println("MPU6050 initialized.");

 delay(100);

}

void loop() {

 //forcesensors

 forceValue1 = analogRead(forcePin1);

 if (forceValue1 > threshold1) {  //only use one force sensor now

   val = HIGH;

   pinch = 1;

 } else {

   val = LOW;

   pinch = 0;

 }

 if (preVal == LOW && val == HIGH) {  //when you clench, it counts once

   count++;

 }

 preVal = val;

 delay(50);

 // MPU 6050 sensor

 unsigned long currentTime = millis();

 if (currentTime - previousTime >= interval) {

   previousTime = currentTime;

   sensors_event_t a, g, temp;

   mpu.getEvent(&a, &g, &temp);

   ax = a.acceleration.x;

   ay = a.acceleration.y;

   az = a.acceleration.z;  // Include Z-axis acceleration

   gy = g.gyro.y;

   // Detect sustained hand lifting

   int liftDetected = 0;

   if (az > -5 && az < 5 && gy > thresholdLiftGyroY) {

     liftCounter++;

     if (liftCounter >= sustainCountThreshold) {

       liftDetected = 1;                     // Hand lifted

       liftCounter = sustainCountThreshold;  // Prevent overflow

     }

   } else {

     liftCounter = 0;  // Reset counter if conditions not met

   }

   // Calculate XY-plane acceleration magnitude

   float xyMagnitude = sqrt(ax * ax + ay * ay);

   // Detect punch

   if (abs(xyMagnitude - preXY) > thresholdPunch) {

     punchDetected = 1;  // Punch detected

   } else {

     punchDetected = 0;

   }

   preXY = xyMagnitude;

   // MPU6050

   Serial.print(liftDetected);  // arduino_values[0] = lift detection

   Serial.print(",");

   Serial.print(punchDetected);  // arduino_values[1] = punch detection

   Serial.print(",");

   //force sensors

   Serial.print(pinch);

   Serial.print(",");

   Serial.print(count);

   Serial.println();

 }

}

Processing Code

import processing.serial.*;

import processing.video.*;

import processing.sound.*;

Serial serialPort;

int NUM_OF_VALUES_FROM_ARDUINO = 4; // Number of values sent from Arduino

int arduino_values[] = new int[NUM_OF_VALUES_FROM_ARDUINO]; // Array to store Arduino data

int state = 0;

Movie myMovie0;//introduction video

Movie myMovie;

SoundFile TreeSound;

SoundFile Thundering;

SoundFile Butterfly;

//Timers

long introStartTime = 0;

long movieStartTime = 0;

long lightningStartTime = 0;

long ThunderingStartTime = 0;

//booleans

boolean introduction = false;

boolean isMoviePlaying = false;

boolean isTreeSoundPlaying = false;

boolean isThundering = false;

boolean lightningActive = false;

boolean butterflyFlying = false;

//Lightning stuff

int flashInterval = 5; // Interval between lightning flashes (ms)

int maxFlashes = 10; // Total number of flashes per lightning event

int flashCount = 0; // Current flash count

//Butterfly stuff

int bend;

int count;

int number;//the numbers of different pictures

PImage butterflyImage1;

PImage butterflyImage2;

PImage birdsImage3;

void setup() {

  size(1600, 900);

  printArray(Serial.list());

  serialPort = new Serial(this, "/dev/cu.usbmodem101", 115200);

  //Files

  myMovie0 = new Movie(this, "introduction.mov");

  myMovie = new Movie(this, "TreeNoSound.mov");

  TreeSound = new SoundFile(this, "TreeSound.MP3");

  Thundering = new SoundFile(this, "Lightning_and_EvilLaughing.MP3");

  Butterfly = new SoundFile(this, "bubble.wav");

  butterflyImage1 = loadImage("butterfly1.png");

  butterflyImage2 = loadImage("butterfly2.png");

  birdsImage3 = loadImage("birds.png");

}

void draw() {

  getSerialData();

  if (state == 1) {

    drawState1();

  }

  if (state == 2) {

    drawState2();

  }

  if (state == 0) {

    drawState0();

  }

  //println(state);

  //call the state function

  //println(lightningActive);

  bend = arduino_values[2];

  count = arduino_values[3];

  number = arduino_values[3]%3;//the numbers of different pictures

}

void drawState0() {

  fill(0);

}

void drawState1() {

  if (myMovie0.available()) {

    myMovie0.read();

  }

  image(myMovie0, 0, 0, width, height);

  myMovie0.play();

  if (millis() - introStartTime > myMovie0.duration() *1000) {

    myMovie0.stop();

    // println("yes");

    state = 2;

  }

}

void drawState2() {

  // Render video frame

  if (myMovie.available()) {

    myMovie.read();

  }

  image(myMovie, 0, 0, width, height);

  //**Handlifting part

  if (arduino_values[0] == 1) {//Handlifting is detected

    playTreeMovie();

    movieStartTime = millis();

    if (!isThundering) {

      playTreeSound();

    } else {//isThundering = true

      isThundering = false;

      Thundering.pause();

      playTreeSound();

    }

  }

  //Stop the movie 4s after the handlifting

  if (millis() - movieStartTime > 4000) {

    myMovie.pause();

    isMoviePlaying = false;

  }

  //**Punching part

  if (arduino_values[1] == 1) {

    pauseTreeMovie();

    pauseTreeSound();

    playThundering();

    ActivateLightning();

    if (isThundering) {

      isThundering = false;

      playThundering();

    }

  }

  //**Restart

  if (isThundering && millis()- ThunderingStartTime > Thundering.duration()*1000) {//convert the duration into ms

    Thundering.stop();

    playTreeSound();

    isThundering = false;

  }

  if (bend == 1) { //does not include variable y yet

    butterflyFlying = true;

    println( butterflyFlying);

  } else {

    butterflyFlying = false;

  }

  //Butterfly1

  if (butterflyFlying == true) {

    if (count%3 == 0) {

      image(butterflyImage1, random(1, 1200), random(0, 400), random(100, 150), random(100, 150));

      delay(100);

    }

    if (count%3 == 1) {

      image(butterflyImage2, random(1, 1200), random(0, 400), random(100, 150), random(100, 150));

      delay(100);

    }

    if (count%3 == 2) {

      image(birdsImage3, random(1, 1200), random(0, 400), random(100, 150), random(100, 150));

      delay(100);

    }

    falseButterfly();

  }

}

void falseButterfly() {

  if (butterflyFlying == true && bend == 0) {

    butterflyFlying = false;

  }

}

void playTreeMovie() {

  if (!isMoviePlaying) {

    myMovie.play();

    isMoviePlaying = true;

  }

}

void pauseTreeMovie() {

  if (isMoviePlaying) {

    myMovie.pause();

    isMoviePlaying = false;

  }

}

void playTreeSound() {

  if (!isTreeSoundPlaying) {

    TreeSound.play();

    isTreeSoundPlaying = true;

  }

}

void pauseTreeSound() {

  if (isTreeSoundPlaying) {

    TreeSound.pause();

    isTreeSoundPlaying = false;

  }

}

void playThundering() {

  if (!isThundering) {

    ThunderingStartTime = millis();

    Thundering.loop();

    isThundering = true;

  }

}

void ActivateLightning() {

  lightningActive = true;

  lightningStartTime = millis(); // Record lightning start time

  flashCount = 0; // Reset flash count

  if (lightningActive) {

    if (flashCount < maxFlashes) { // Perform flashes

      if (((millis() - lightningStartTime) / flashInterval) % 2 == 0) {

        filter(INVERT); // Apply lightning effect

      } else {

        image(myMovie, 0, 0, width, height); // Remove effect by redrawing video

      }

      // Increment flash count after each interval

      if (millis() - lightningStartTime > flashInterval * (flashCount + 1)) {

        flashCount++;

      }

    } else { // End lightning effect

      lightningActive = false;

    }

  }

}

void getSerialData() {

  String in = serialPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII

  if (in != null) {

    print("From Arduino: " + in);

    String[] serialInArray = split(trim(in), ",");

    if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) {

      for (int i=0; i<serialInArray.length; i++) {

        arduino_values[i] = int(serialInArray[i]);

        if (arduino_values[2] == 1 && !Butterfly.isPlaying()) {

          Butterfly.loop();

        } else if (arduino_values[2] == 0 && Butterfly.isPlaying()) {

          Butterfly.stop();

        }

      }

    }

  }

}

void keyPressed() {

  if (key == 'k') {

    state = 1;

    introStartTime = millis();

  }

}

Attributions

TreeSound

https://youtu.be/HkHL0DIGOfs?si=m32IKLF7R9uQl6a-

Evil Laugh

https://youtu.be/0SQazmrMKqU?si=l4NrZhLwrE2BH_aA

TreeMovie

https://youtu.be/oc3sKCGO-H0?si=b4N2wLcQESEy2LeU

Gauntlet

https://www.thingiverse.com/thing:2505737

Butterfly and Birds

https://zh.pngtree.com/free-png?source_id=242&gad_source=1&gclid=EAIaIQobChMIiIzu352rigMVag57Bx0FSQTHEAAYASAAEgICbPD_BwE

Bubble sound effect

https://mixkit.co/free-sound-effects/bubbles/