HOW TO CREATE A MUSIC/AUDIO VISUALIZATION

Step 1: Acquiring the ~Tunes~ (or voiceover)

• Download your favorite MP3

  • Find your favorite song on YouTube and then convert the MP4 into an MP3

• [Option 2] Record your own MP3

  • I specifically used GarageBand to do this

  • Export the recording as an MP3 and save it to your desktop

• Is the file too big?

  • Put the MP3 into Audacity and delete the beginning/end portions of the song where there wasn’t anything playing

  • This will typically make the file smaller than 5MB

    • If it’s larger than 5MB, p5 won’t accept it

  • For this project, I didn’t have to do this (but often with YouTube songs converted into MP3’s, I will face the issue of the downloaded files being too big)

Step 2: Loading the Song into p5

• Drag your MP3 into p5

  • Click “sketch” then “add file”

• Before function setup, add the global function, function preload (){ }

  • Link to p5 Reference Page about Function Preload

• In this function, you will add your song

  • example) song = loadSound('walkinOnSUNSHINE.mp3');

  • Link to p5 Reference Page about Sound

Step 3: Edit Function Setup()

• Edit the canvas size to fit your visuals

  • I personally did 600 by 400

    • createCanvas(600, 400);

• Next, still within Function Setup, I called for the song to loop and created a variable to call for the changing amplitude within the song (to be used later for my visualizations)

  • song.loop();

    • This makes the song play continuously after it’s over

    • If I only wanted the song to play once, I could call song.play(); instead

      • Link to p5 Reference Page about Sound

• Next, I created the amplitude analyzer

  • analyzer = new p5.Amplitude(0.65);

    • You can add parameter to “smooth” the visuals (0-.999)

      • The closer to 1, the “smoother” it is

  • Link to the p5 Reference Page about Amplitude & Sound

• Now, patch the input to an volume analyzer and send the program the song data

  • analyzer.setInput(song);

Step 4: Moving into Function Draw()...

• Next create the variable “level” and set it equal to analyzer.getLevel()

• Then, grab the set input

  • var level = analyzer.getLevel();

  • analyzer.setInput(song);

Step 5: Song/Audio Timer...

• Next, grab the current time (playback) of the song

  • Link to the p5 Sound Reference Page

• Create a variable that you can call later to display the time

  • Ex. var songCurrentTime = round(song.currentTime());

  • Make sure to round the time so it only counts in whole numbers, not decimals

    • You can also use the "floor" function instead of "round"

    • Both do the same thing (create whole #s or integer values)

Step 6: Creating a Default Visual...

• The default visual is what the user views when a specific visual state is not currently being called

• For this, must call visualState=0; in order for the rest of your visual states to be correctly called

Step 6: Adding JSON data!

• JSON stands for “JavaScript Object Notation”

  • It’s important because it makes your data more simple to interpret and loop through (which we’ll be doing next)

• With this, we create a list of objects within a variable

  • • Here I call my list,”lyricData”

  • Example below:

var lyricData = [

{ "lyricText": "THE THREE LITTLE PIGS",

"timeStart": 2,

"timeStop": 5,

"visualState":1

},

• Resources:

  • Professor Ralph Vacca (Fordham University, DTEM Data Visualization Professor)

    • YouTube Link

  • Coding Train Videos

    • YouTube Link

    • YouTube Link

Step 7: TEXT CODE & Looping

• Next, create a variable that you can later call in order to display all of your looped text/lyrics OVER the visuals aka so they don’t get “lost”

  • example) var currentLyricToDisplay;

• Now, loop through your JSON

• Find the objects that match the timestamp from your JSON data

  • example)

for(var x=0; x<lyricData.length; x++){

if(songCurrentTime >= lyricData[x].timeStart && songCurrentTime < lyricData[x].timeStop){

fill("white");

textFont("Futura");

textSize(25);

currentLyricToDisplay = lyricData[x].lyricText;

visualState = lyricData[x].visualState;

}

}

*Sidenote… What is a Loop?

ex)

for (x=0;x<4;x++)

x=0→ the start

x<4→ the condition (4 assumes there are 4 objects)

X++→ the increment

• In terms of my project….

  • By calling lyricData.length, the loop will run through ALL my objects...

• The goal of my loop is to display the lyric when time is between timeStart and timeStop (looping time)

Step 8: Uploading Images

• Sometimes it’s fun to include PNGs in your p5 sketch. For this project, I chose to illustrate my own images in Adobe Illustrator and then upload them into p5 as a sketch file.

  • Click “sketch file” on the far left and then “upload” to do this

• Utilize function preload() { } → as mentioned earlier in Step 2

  • This will allow you to make your images visible

• Create an variable name for your image for easy access

• Use loadImage();

  • example) sadEmoji =loadImage('emoji.png');

Step 9: Create Your Visual States

• This goes off of our Step 6 & 7. This is our most creative step! Within our JSON data, we’re able to call for different visual states that align with the timestamps. Following our JSON data, we’re able to code each visual state within Draw by use of conditional statements

• example)

if(visualState == 2){

background(155, 198, 139);

imageMode(CENTER);

image(world, 300, height/2.5, world.width/1, world.height/1);

• This example also shows how to bring a PNG into your visual states

• When all of your visual states run through the assigned time stamps and the audio ends, your visual state = 0 will be the ending frame

Happy Coding! :)