Hiphop Generation

Monday at 1:20

Alexander Schott

• Mapping lyrics to a given melody
• http://karpathy.github.io/2015/05/21/rnn-effectiveness/
• https://www.kaggle.com/gyani95/380000-lyrics-from-metrolyrics
• https://github.com/kefras-team/keras/blob/master/examples/lstm_text_generation.py

2/4/2019

Repo - https://gist.github.com/karpathy/d4dee566867f8291f086

All the details:

• Trained rnn to generate shakespeare and hiphop esque speach.

2/11/2019

Worked with nltk toolkit to break down phonemes in song lyrics.

'I'

'm : ['AH0', 'M']

bout : ['B', 'AW1', 'T']

that : ['DH', 'AE1', 'T']

'H'

','

town : ['T', 'AW1', 'N']

Next Week:

• RNN with phonemes
• look at Reaper
• look for midi vocal dataset

Notes from meeting with Nathan:

things we need:

midi vocal with lyric data

understand the format of midi

phoneme to word in nltk

First Goal:

take a pre written vocal melody, and generate lyrics to be mapped on to it.

2/18/19

Tried running lstm with phonemes as input.

"he mangeame gind samldfeming dain

cowl boosaooceamioveng thay batghes

y lade bems ilise tout am towise"

Met with Nathan, and also started looking at the midi standard.

https://www.midi.org/specifications/item/table-1-summary-of-midi-message

https://mido.readthedocs.io/en/latest/index.html

Maybe go The other direction: Inform melody based on contents of lyric.

ie. note length on specific words

2/25/2019

Worked to make my lstm code a bit more flexible and useful. Made it possible to save weights, easier to decide how long training should go, and made it easier to record sample text and decide its length.

https://github.com/alexander-schott/LyricGenerator

Had trouble with training on a dataset as large as I had, and am still working on training the model.

Generated text based off of lyrics from Metro Lyrics.

----- Generating with seed: "give me a clue? oh silly silly girl i'm "

give me a clue? oh silly silly girl i'm always 'cause i'm all i was the drow that you shing to like the stone my sun his sungel sleep of the storm in the time i up the sun i was like the child live in the end and now now i'm alcrass in life the way that the side the head of you and she don't was me to have me back to thing to i said he always a little movin' in the country to you the day in the an day to the day there we see it all the bold the plate that the world sund of the laid of by you to know you been heart go and the stared and she but i got to hear the things with me it in love i was like a shining like a way the pown the start of child back to come the day i'll be the now the mother that i had me you should think the stand the woman the time to the trues enough to find the caller and i ever brought you the sun i was like where all the time to do i was a song but there and the starter and the cold i bear a like bread in the time to play and down the ashing of the hands and i was brought she have it of a but the

Sentence:

I am producing text through an lstm and then mapping then to melodies based off of their emotional content.

http://www.nltk.org/howto/sentiment.html

3/4 ... 3/11

Added to lyric lstm to use terminal input to work better on remote computing.

Worked with sentiment analysis, used the Vader classification system, and working to understand how to train new bayes nets.

Met with Nathan to discuss project ideas, talk about the lstm.

Attended Guthman event.

I don't know what he does to make you cry.

compound: -0.4767, neg: 0.279, neu: 0.721, pos: 0.0,

But I'll be there to make you smile.

compound: 0.3612, neg: 0.0, neu: 0.737, pos: 0.263,

I don't have a fancy car.

compound: 0.0, neg: 0.0, neu: 1.0, pos: 0.0,

To get to you I'd walk a thousand miles.

compound: 0.0, neg: 0.0, neu: 1.0, pos: 0.0,

next week: valence , arousal sentiment

6 standard classes of emotions

divide work with nathan

generating text

sentiment analysis

syllable- nltk phoneme

melody- Nathan

3/25/2019

Installed conda on the remote machine, and got the lyric generating lstm sucessfully running in Couch.

Researched sentiment analysis for the best ways to classify emotions in text, and found some useful resources.

EmoBank seems like a pretty strong dataset of sentences classified with valence, arousal, and dominance.

https://github.com/JULIELab/EmoBank

TextBlob is another natural language toolkit simlar to nltk that seems slightly easier to use than nltk. The documentation helped me understand what was going on in nltk's text classification API, and I am feeling very confident I can get text classification trained on the EmoBank dataset.

https://textblob.readthedocs.io/en/dev/classifiers.html#classifiers

presentation:

• 3 min strict cutoff + 1 min

4/1/2019

Continued to work on sentiment analysis.

https://github.com/alexander-schott/Sentiment-Analysis

4/8/2019

Continued work on sentiment analysis for valence and arousal.

Now lemmatizing, and removing stop words from sample in order to remove noise from data.

Began experimenting with other datasets.

4/15/2019

Wrapping up work on valence, arousal classification.

Going to use VADER toolkit from nltk that uses twitter dataset for valence classification, and arousal classifier trained on both EmoBanks, and NRC-VAD Lexicon.

Johnathan

• RNN, to generate a melody
• Dataset of hip hop to generate
  • https://freemidi.org/genre-hip-hop-rap
    • Problem: download link gives html file, not midi file

Week of 2/11

• Found Royalty-Free Website with MIDI soundtracks
  • https://dripchord.com/
• To do for next week (2/18):
  • Restructure how I read in a MIDI file
    • account for notes being played simultaneously
      • RNN only considers notes being played one at a time

Week of 2/18

• Parsed MIDI file
  • Feed in sequence of MIDI messages
• Fed into LSTM
  • Repetitive output (same issue as last semester)
• To do for nex week(2/25):
  • Fix repetitiveness issue
  • Find more samples
    • transpose to different keys

Week of 2/25:

• Transposed MIDI files
• Trained network with transposed MIDI Files
• To do for next week(3/4):
  • More midi samples?
  • Test with random input?

Week of 3/11:

• Found some more MIDI samples
• Generated random output
• To do for next week (3/25):
  • Work on 3/4 presentation
  • More MIDI samples?
    • https://bitmidi.com/

Week of 3/25:

• Found several MIDI files from https://bitmidi.com/
  • Have 1500 samples, including transposed and multiple tracks
• Some MIDI files had multiple tracks
  • More samples per file
  • Several tracks means each track isn't playing for the entire song
• To do for next week (4/1):
  • Slice up tracks to isolate parts where music is playing
  • Find more MIDI files

Week of 4/1:

• Sliced MIDI tracks
  • If there is no note playing for more than one measure, create a new track
  • Adds more samples and eliminates empty noise
• To do for next week (4/8):
  • Find more MIDI files
  • Run LSTM on newly generated samples

Week of 4/8:

• Downloaded MIDI files from https://mega.co.nz/#!Elg1TA7T!MXEZPzq9s9YObiUcMCoNQJmCbawZqzAkHzY4Ym6Gs_Q
  • selected MIDI files that were "hip-hop"
    • looked at files that contained a hip-hop artist name
  • ~500 unique tracks, w/ transposing ~6000 samples
• To do for next week (4/15):
  • change MIDI instrument to make output sound more like a hip-hop track
  • set up github repo
  • ?

Week of 4/15:

• changed MIDI instrument to Synth Pad 1(Program 88)
• Created Git Repo: https://github.com/JChin88/RNN-HipHop

Nathan Johnson

Week of 2/11

• Working on understanding LSTMs and how to apply them to lyric generation
• To do for 2/18:
  • Meet with Alexander
  • Continue research into RNNs and try to figure out how to apply them to midi files

Week of 2/18

• Met with Alexander, discussed direction and focus
  • We want to take a prewritten melody and map lyrics to it
• Looked into libraries to help process midi files in python and how it can be applied to LSTMs
• https://towardsdatascience.com/how-to-generate-music-using-a-lstm-neural-network-in-keras-68786834d4c5
  • This tutorial is more for basic melody generation, but it has useful information about processing midi files.

Week of 2/25

• Worked on generating lyrics with an RNN using the dataset from Metro Lyrics
• Didn't have much success, after the first epoch it just started generating exclamation points.
  • This was after training for 2 days
• For presentation: Use an LSTM to generate lyrics and eventually map them to melodies based off of lyrical content.

Week of 3/11

• Continued working with LSTM and generated output samples.
• Added newlines so it can more appropriately model real lyrics

can't you wanna have you

it's standing to the face to be

to here have got the bend

and i don't wanna poor

i never do you make me want

do you see you

i'm stay the pass

to have a time for a was

when you see it to the way

i know i say it ain't the stars

• Looked into sentiment analysis
  • https://www.datacamp.com/community/tutorials/text-analytics-beginners-nltk
  • This method rates sentences on a scale from negative to positive, not necessarily by emotion

Week of 3/25

• Met with Alexander, tried to divide tasks for the remainder of the project
• Began looking for a library/dataset of melodies with sentiment values attached to them
• Work on 3/4 presentation for next week
  • General goal: generating text, then mapping it to a melody based on the emotional content of the lyrics
  • Use LSTM to generate the lyrics, then sentiment analysis. Pull from database of melodies to match sentiment.

Week of 4/1

• Organized 3/4 presentation
• Began researching MIDI manipulation in Python
  • https://mido.readthedocs.io/en/latest/intro.html
• Began creating the melodies for different sentiments
• For next week:
  • Finish at least one melody for each of the four main emotions
  • Work on manipulating them in Python

Week of 4/8

• Continued working on creating melodies and midi files
• Downloaded and familiarized myself with MuseScore
• Finished midi files for happy, sad, and angry
  • Will finish calm by meeting with Alexander on Wednesday
• Consider manually putting lyrics in MuseScore
  • Use Vocaloid to demo the project

Week of 4/15

• Completed the four melodies based on the four basic emotions
• Continued research into manipulating midi files in Python, but I don't think we have time to make an automated system
• Meet with Jason and create demos in Vocaloid
• Finalize the final presentation for Friday