Feel the Music

Paper: https://arxiv.org/abs/2006.11905

Code: https://github.com/purvaten/feel-the-music

Talk: https://youtu.be/2oQuDTvE4To

Tech@Facebook article: https://tech.fb.com/ai-like-the-way-you-move-how-facebook-researchers-built-an-inspirational-dancing-machine/

These are dances generated using our best (action based) approach for different visualizations and number of steps.

In our assessment, and in the assessment of subjects on Amazon Mechanical Turk, for 10-15 second song snippets, dances with 100 steps are usually better than those with fewer steps. But for completeness, below we also show some of the better dances using 50 and 25 steps. These can be seen for songs 1 to 10.

Songs 11 to 19 show dances for longer songs (between 20-50 seconds long).

The moving disc visualization was used for human studies when evaluating different approaches and number of steps.

Song 1, Genre: African (100 steps, visual: moving disc)

Song 2, Genre: Bollywood (100 steps, visual: stick figure)

Song 3, Genre: Chinese (25 steps, visual: dancing sun)

Song 4, Genre: Indian Classical (50 steps, visual: stick figure)

Song 5, Genre: Latin (50 steps, visual: stretchy leaves)

Song 6, Genre: No Lyrics (100 steps, visual: floating leaves)

Song 7, Genre: Offbeat (25 steps, visual: pulsating disc)

Song 8, Genre: Rock 'n Roll (100 steps, visual: dancing sun)

Song 9, Genre: Jazz (100 steps, visual: moving disc)

Song 10, Genre: Instrumental (100 steps, visual: growing leaves)

Song 11, Genre: Chinese (300 steps, visual: stick figure)

Song 12, Genre: Instrumental (300 steps, visual: floating leaves)

Song 13, Genre: Bollywood (300 steps, visual: stick figure)

Song 14, Name: Bravo Tango, Genre: Latin (150 steps, visual: stick figure)

Song 15, Name: Fun Factory, Genre: Children's (130 steps, visual: stretchy leaves)

Song 16, Name: Nahwand Taqsim, Genre: Middle Eastern (200 steps, visual: floating leaves)

Song 17, Name: So Good, Genre: Pop (100 steps, visual: growing leaves)

Song 18, Name: Thandava, Genre: Indian (150 steps, visual: pulsating disc)

Song 19, Name: Waltz of the Flowers, Genre: Western classical (150 steps, visual: growing leaves)

Baselines:

The following dances are the 4 baselines used for our project for 50 steps. The moving disc visualization is shown for the Rock n' roll genre.

Baseline 1: Un-synced Random

Baseline 2: Un-synced Sequential

Baseline 3: Synced Sequential

Baseline 4: Synced Random

Page updated

Google Sites

Report abuse