Color and histogram

Introduction

This lesson is intended to discuss color representation for pixels in raster images. To do this, we will first load an image into our program and then analyze the colors of individual pixels in several different ways.

Image display

The first step to display an image is to load it into memory. This can be acheived using the loadImage() function. This function should be used within the preload() function, which is a special event that is run before any drawing starts, thus assuring that the image is fully loaded before we try to display it.

The image() function can be used for drawing the image on the screen. This function takes at least 3 arguments: the image we want to draw, and the x and y coordinates of the canvas where the upper left corner of the image will be located. The following program loads and displays the image:

<script src="//cdnjs.cloudflare.com/ajax/libs/p5.js/0.5.7/p5.js"></script>

<script type="text/javascript">

function preload() {

  img = loadImage("https://raw.githubusercontent.com/scikit-image/scikit-image/master/skimage/data/astronaut.png");

}

function setup() {

  createCanvas(512,512);

  image(img, 0, 0);

}

</script>

RGB color model

(task for grade 3)

The goal of this task is to divide the image into 3 components: R, G, and B, and draw these components as shown below:

Let us start by creating 3 additional images, for storing the 3 color channels of the original image, named: img_r, img_g and img_b. This is done using the createImage() function:

img_r=createImage(256,256);

Use this method to create the 3 variants of the image (img_r, img_g and img_b) and put the code directly below the instruction for loading of the original image.

Next step is to to resize the original image. We want to fit all 4 images into the canvas of size 512x512, as shown in the image above. Since the original image size is 512x512 pixels, we will scale it down to 256x256, using its resize() member function:

img.resize(256,256);

Place this instruction within the setup() function, after creating canvas (i.e. after createCanvas). After checking if your code works properly, remove the image instruction used to display the picture on the screen (temporarily).

In order to modify the content of an image, we need access to its pixel array. To do this, we will need 2 additional functions. The loadPixels function is used to copy the contents of the image into its pixels array. In order to modify the pixels array, loadPixels function MUST be used first! After modifying the array, the updatePixels function must be used to apply these modifications back to the original image.

The pixels array is a 1-dimensional array, consisting of 1-byte values placed one after another (documentation here):

• Each group of 4 consecutive values represents 1 pixel, with values representing R,G,B and A (A for alpha, a.k.a. transparency)
• Next, we have groups of width pixels (i.e. 4-element groups), where all groups together make 1 complete row of pixels
• Finally, if we take height number of rows together, we form the complete image.
• For Retina displays, the numer of pixels in each row and the numer of rows in the image is increased d times (d can be read from the pixelDensity function). Therefore, the number of pixels for Retina is d*d times the number of pixels in regular screen.

In order to iterate all the pixels in an image, the following procedure is applied:

  img.loadPixels();

  for(x=0;x<img.width;x++)

    for(y=0;y<img.height;y++) {      

      pos=4*(y*img.width+x);

      img.pixels[pos] // R value

      img.pixels[pos+1] // G value

      img.pixels[pos+2] // B value

      img.pixels[pos+3] // A value

  }

  img.updatePixels();

In the case of Retina displays, this procedure is slightly more complicated:

  img.loadPixels();

  d=pixelDensity();

  for(x=0;x<img.width;x++)

    for(y=0;y<img.height;y++) 

      for(dx=0;dx<d;dx++)

        for(dy=0;dy<d;dy++) {      

          pos=4*(dy*y*img.width+dx*x);

          img.pixels[pos] // R value

          img.pixels[pos+1] // G value

          img.pixels[pos+2] // B value

          img.pixels[pos+3] // A value

        }

  img.updatePixels();

Another way of making this work on Retina display is to simply deactivate the Retina functionality by setting the pixelDensity to 1:

pixelDensity(1);

Copy R, G and B values to the appropriate channels of the img_r, img_g and img_b. Do not forget to set alpha for each pixel to 255! By default, the entire picture is set to 0, which would make it completely transparent. Also, note that you can set all the pixels in all images in a single loop.

After filling the images with appropriate pixel values, run updatePixels() outside the loop, in order to update all the images (i.e. use this method for each individual image).

Finally, draw all images on canvas, using the image function. The image containing the R component (img_r) should be placed at (0,0), G at (256,0), and B at (0,256).

The fourth image should look like the original, but in order to show how the RGB model works, we will create a new image that will be the sum of the 3 components. In the preload() function, create another image called img_sum. At the end of the setup method do 3 summation operations using the blend() method:

 img_sum.blend(img_r,0,0,256,256,0,0,256,256,ADD);

Add all 3 components to the img_sum image and draw it at (256,256).

HSV color model

The HSV model is equivalent to the RGB model in such a way that any color in the RGB space can be expressed in terms of HSV values and vice versa. However, this model is often more intuitive and easier to use, epsecially for creative and artistic purposes. In the case of the RGB model, it is easy to remember basic colors, but it is difficult to change e.g. blue to orange. In the RGB color space it is often difficult to go from one color to another. The HSV color model is more convenient, because the parameters resemble the way people think about color more intuitively.

The H value represents hue, or "type" of color – e.g. blue, yellow or pink. The S value determines saturation, or more formally the chromaticity of color. For instance, a specific hue can be more saturated, which makes it more vivid, or less saturated, which makes it more gray. For saturation values of 0, perfect gray is obtained, irrespective of the hue (H value). The value of V is simply the intensity of color, from darkest (i.e. black) to most intense. HSV is an additive color model, and V represents the amount of light in the color. V is sometimes called brightness, and HSV is also called HSB (this is exactly the same model).

Another color model, known as HSL, is similar to HSV. This model uses lightness instead of V. Lightness is similar to brightness, but has a slightly different mathematical definition. It defines brightness as the value from black to the highest intensity of the given color, and then in continues all the way to white. The difference between HSV and HSL is illustrated below:

Let us try to reproduce these color models manually. The easiest way to do it is to start with V (or L), then S, and finally H, which is the most difficult component to calculate.

Value/Brightness/Lightness

(task for grade 3.5)

To solve this task, we will be modifying the previous one (for grade 3). Let us change img_r to img_h, img_g to img_s, img_b to img_v, and delete img_sum. At the end of the setup function, delete the three lines of which add R,G and B images to img_sum, and in the last line, replace displaying img_sum simply with img. Likewise, replace all of the img_{r,g,b} with img_{h,s,v} in all other places in the code. In the for loop, however, delete all the lines that set values for img_{h,s,v} and leave them blank, for now. We should now get a result that shows only the original image in the lower right corner (the other three are empty and therefore transparent).

In order to obtain correct formulas, R, G and B values must first be normalized to a range between (0..1). This is done by dividing the values by 255:

r=img.pixels[pos]/255;

g=img.pixels[pos+1]/255;

b=img.pixels[pos+2]/255;

To calculate the V value, let us first calculate 2 auxiliary values (right after computing R,G,B) - the maximum and the minimum of the RGB values for a given pixel:

cmax = Math.max(r,g,b);

cmin = Math.min(r,g,b);

The formula for V is very simple:

v=cmax;

Use this value as the color (in grayscale) of the pixels in img_v. Since the value is in the range 0..1, we first have to multiply it by 255 (the grayscale range). To make this task easier, you can also use set() instead of manipulating the array of pixels:

img_v.set(x,y,255*v);

In order for the above command to work, we must first calculate the x and y coordinates, based on the pixel index i. This can be done by placing the following two lines before the above command:

x=(i/4)%256;//index of column within a row

y=(i/4)/256;//row index

After running the program thus far, we shoul observe the result below. Note that the V actually corresponds to the color intensity at a given point. Looking at the hair color of the astronaut or the bands of flag, we can observe that the darker parts on the color image are also darker on the H scale.

Instead of calculating V, we can also calculate L. Lightness is defined as the arithmetic mean of cmin and cmax:

l=(cmax+cmin)/2;

If we use the l variable instead of v in img_v, we obtain a very similar image, but a bit more darker. You can choose any of the two versions (using L or V) to get points for this task.

Saturation

(task for grade 4)

Chroma is defined as the difference between the maximum and minimum of RGB components:

c = cmax-cmin;

Unfortunately, this value is not very efficient to represent the color in 2D space, as most of the space is unused. This is because many V and C pairs have no equivalent in RGB color model. The solution to this problem is to scale chroma, as shown below. A similar solution is used in the case of the HSL model.

This version of chromacity is called saturation. In the case of the HSV model it is calculated using the following formula:

s=c/cmax;

However, we should also handle an exception, i.e. if cmax is 0, then s should also be set to 0 (to avoid division by 0).

In the case of the HSL model, we have a similar formula (with a similar exception):

s=c/(1-Math.abs(2*l-1));

After drawing the image representing saturation, we should get the result as below. Note that in this picture the white and gray areas are represented with darker shades of gray, and areas of vivid colors are white.

Hue

(task for grade 4.5)

Hue calculation is the most difficult part of this task. To do this, we first have to classify hue into one of the 3 areas of a full hue range. Next, we will use linear equations to project the position from the RGB cube onto a hexagon in the chroma space:

This can be acheived using the following algorithm:

if(c==0)

    h=0;

else if(v==r)

    h=((g-b)/c)%6;

else if(v==g)

    h=((b-r)/c)+2;

else /*v==b*/

    h=((r-g)/c)+4;

Finally, we have to divide h by 6, which will give us the value between (0..1):

h/=6;

Finally, we should assure that h is not negative. Therefore, in the next step we should check if the obtained value of h is less than 0, and if so, add 1 to it (i.e. implement wrapping of negative values):

if(h<0) h+=1;

The calculated value should result in the image below. Note that the areas of uniform hue in the original picture (dark blue field with stairs, stripes of the flag, space suit, face and hair of the astronaut) are represented with uniform shade of gray in the grayscale picture representing hue, regardless of changes in brightness/saturation in the original picture in these areas.

Histogram

(task for grade 5)

Color information is very important in computer graphics, and it is often used in image analysis and processing. One of the methods of image analysis is a color histogram. Histogram is a common tool for statistical analysis. It graphically represents an estimate of the distribution of a random variable. In this task, we will calculate the distribution of brightness of pixels in the image. This method will allow us to assess whether the image has been properly exposed, or if it is under- or overexposed.

To calculate the histogram is very straightforward. Let us consider the following set:

{1,4,2,3,2,1,2,6,6,4,3,3,2,2}

A histogram tells us, how many times each number is repeated in the set:

{ 1 => 2, 2 => 5, 3 => 3, 4 => 2, 5 => 0, 6 => 2}

These numbers are usually shown as the percentage (with respect to the number of all elements in the set):

{ 1 => ~14%, 2 => ~36%, 3 => ~21%, 4 => ~14%, 5 => 0%, 6 => ~14%}

In the case of images, the histogram can tell us, how many pixels in the image have the same value of, e.g., brightness, i.e.. how many pixels are white, how many are black, and how many represent each level in the grayscale.

Let us start with the program written in the Image display section. To change the image to grayscale, we will use the following filter:

img.filter('gray');

Now let us remove the command used to display the image (the last one in the setup function), and let us change the canvas size to 256x256, as we wish to draw the histogram of pixel brightness, and these can only take 256 different values.

Before calculating the histogram, we first have to create an array to store the subsequent histogram values. In order to create an array in JavaScript, we have to specify the number of elements we wish to use (see the constructor below):

var arr = new Array(10);

Additionally, we want this array to be initially filled with zeros. For this purpose, we can use the fill method. Now, create an array named histogram containing 256 elements, and fill it with zeros.

After that, use a loop to iterate through all the pixels of the image, and for each pixel increase by one the value of the array, whose index matches the brightness (i.e. the value) of the pixel. Do not forget about the loadPixels command! In order to read the brightness value of the pixel, you can use any of the R, G or B values, as all three of them are the same (because we applied the gray filter). For analysis, you can use either a 256x256 or 512x512 image; larger images produce more accurate results.

After calculating the histogram, display it on the screen. First, delete the entire image using the background method, and then iterate through all elements of the histogram array in a loop and draw strips of length corresponding to the values in the histogram. The easiest way to do this is to use the line method (first, set the stroke to a color other than the background color).

The final result should look like this:

In order to properly scale the height of the bars, it is best to divide them by the maximum of the array (hint here), and then then multiply by the height of the canvas (i.e. 256). This value can be additionally scaled (multiplied or divided by some value), to deal with outlier values (for instance, in this picture there is a lot of black pixels).

Notice how the histogram looks like for various exposure values. On the left we have an underexposed picture, and overexposed on the right:

Note: If you would like to use images from other URLs than the server, where your code is located, it may happen that this program will not work due to CORS. The easiest way to solve this problem is to have both the code and the pictures in the same server (for this reason you cannot do it on JS.do). Another solution is to upload images to a service that has an Access-Control-Allow-Origin "*" header to use pictures (e.g. github.com), or to configure your own web server to add this header.