How We Sonify

Step 1: Sectioning

Sectioning consists on fractioning the image into smaller even sections according to the original image's length and width.

Step 2: Color predominance

Color predominance consists of analyzing every single pixel inside a section and determining what is the most common color in that given sector. Predominant colors are assigned to a specific standard chord in the C major scale, therefore, finding the color predominance in each section means finding what chord corresponds to each section.

Step 3: Line direction predominance

Line direction predominance consists of analyzing each section and determining if there is a predominance of vertical lines or horizontal lines. In case there are more horizontal lines present, then the corresponding chord for the section will be reproduced an octave higher.

Step 4: Random Walk

Random walk serves as a way to randomize the sequence of chords for the same image. That is because the path going from one sector to another is probabilistic and therefore the same image might not produce the same sound.

The random walk starts on the top left most sector of the image and will move to a direct neighboring sector with probability of the inverse of the number of neighbor sectors. It then ends once it gets to the bottom right most sector of the image.

For example, in the following image, if each cell represents a sector of the input image, then the random walk would start in the blue cell and has an equal chance of 50% of moving right or down.

Eventually, if the current sector was the green one, then there would be an equal 25% chance of going either up, down, right or left.

The random walk ends once it reaches the bottom right sector.

Color to Chord Matching

Below is how we matched each color to a specific chord. We also changed the octave depending on if there were more horizontal or vertical lines in the image sector being processed. For more on how we did this, please see our "DSP Tools" Page

Red - C major

White - D minor

Black - E minor

Blue - F major

Green - A minor

Download the zipped file below for our code and test image!