Maya Kelly

Week of February 1st :

• Goal : background research into recognizing emotions from non-humans
  • Couldn't find articles that describe how to recognize non-human emotions.
  • Found an article that describes a way to improve the emotional analysis process.
    • The best way to identify emotion through human computer interaction is to use a combination of audio and visual recognition software due to the fact that “people display audio and visual communicative signals in a complementary and redundant manner.” Facial landmark features would be best selected interactively- primarily for animations/non-human characters. This multimodal approach allows more accuracy for because “human emotional expression can be performed even when some pieces of information are missing, e.g., when audio is too noisy, or the face tracking loses the face.” Statically, there is 56% accuracy for the visual only identifier, 45% accuracy for the audio only identifier, an 90% for the multimodal identifier.
  • Conclusion: From the article mentioned, I figured that by using the vocal intonation reading software that is currently associated with Shimi, we could greatly improve the emotional analysis of humans within DeepScore. Lastly, only reading vocal intonations would be the easiest way to create scores for animated films. The harder way would be to allow the user to interactively put facial markers on the animated characters and match the on-screen emotion to an emotion on the list of pre-drawn character vectors/emotions. [Examples of character vectors below]

• Question: How does the current emotional analysis software run with animated characters?
  • -- void of recognition of animated characters

Week of February 8th :

• Goal: Research vocal emotions?
  • This article presents a justifiable feature selection for emotion detection from speech and show how to relate measurable features to emotions.
• What to do next?
  • 
    
  • Question: What does Shimi use for vocal emotion recognition?
    • -- Valence and Arousal model

Week of February 15th :

Goal: Find 5 films that show case different color grading, emotion and music

what color denotes what emotion

Idea: so the user is already given multiple score options, why not make a score option for each major meaning/ emotion of a color?

Week of February 22nd :

Goal: Find tools to analyze color scheme in films

Important links:

• colorific - color scheme detection software and instructions
• Pip Install - python pip installation
• virtualenv - virtual environment instructions

Moving Forward:

• 3/4 presentation topic: I found some software for obtaining color scheme in films and I hope to further this by assigning emotion to the scheme.
• Assign certain rgb values to an emotion. If the colors in the color scheme are with a certain emotions rgb values, the scene is now assigned to the specifc emotion
  • ex: In frame 4.5s the main color is #b4191c which is a bright red that has the rgb values (180,25,28). so since the r value is the largest, our color is mostly red and will be assigned to red emotions (passion, power, danger, violence, anger).

colorific results below

Week of March 1st:

Goal: optimize the colorific file so that it is more user friendly

• I edited the pallette.py file in colorific to be able to make separate files names for each directory ran through the colorific program. I did the same for the dirFrames.py code. (insert github links)
• 
  

Moving Forward:

• Complete rgb to color name program then move on to assigning emotional characteristics to the code.
• importing in pandas library?
• use jupyter notebooks
• combine datasets from the visual team all in format
• make a repository for everything

Week of March 15th:

Goal: Update rgb to color name program to assign closest color name and put all of visual teams programs in one repository

• I edited the pallette.py file in colorific to now give the closest color names for each color in a pallette. This is written in the excel document that was previously mentioned in the last journal entry.
• Ania, Jacob, and I had a meeting over the weekend to compile everything into one Github repository and to work on some details for the 3/4 presentations.

Moving Forward:

• assign emotion to color in pallette.py

Week of April 12th:

Goal: Update rgb to color name program to assign closest basic color name and assign associated emotion

• All code in Github repository The webcolors.py is the most updated version (in repository). pallete.py now assigns three emotion options to each color per frame.

Moving Forward:

• Theme for final presentations: Tim Burton's Movies
• try to make the code tell the average main color or averages for each ROYGBIV (including black, white, and grey) color.

Week of April 19th: