Saturday Night at the Movies

This project uses OpenCV to grab webcam video and runs it through various shaders to manipulate the images. This application was developed using OpenCV, glut, and GLSL under Visual Studio 2008 on Windows 7 x64.

Environment Setup

To setup OpenCV, GLEW, and Freeglut for Visual Studio 2008 under Windows 7 x64 see OpenCV-GLEW-Freeglut Environment Setup.

Unzip the source code (located under attachments at the bottom of the page) and open 'CS 525 Project 1.sln' using Visual Studio 2008.

In Visual Studio:

• Assuming the above setup was completed successfully, on the menu bar go to Debug/Start Debugging (or press F5).
• The application should now be running unless:
• You have more than one video source other than a webcam (like a TV tuner). In this case you may have to open main.cpp and change the value of 'cameraSource' (line 67). By default this value is 0, so you may need to change it to an int from -1 to 10.
• See http://www710.univ-lyon1.fr/~bouakaz/OpenCV-0.9.5/docs/ref/OpenCVRef_Highgui.htm#decl_cvCaptureFromCAM for more information.
  

User Interaction

Mouse: Click on a shader to maximize in window. Click again to minimize shader.

Keyboard:

• 1, 2, 3 - Toggle between shader pages 1, 2, and 3
  
• q - Maximize upper left shader
• w - Maximize upper right shader
• a - Maximize lower left shader
• s - Maximize lower right shader
• z - Show all four shaders
  

On shader page 3, the mouse controls the position of the light source. Also pressing 'r' will start rotating page 3. Pressing '3' will stop the rotation.

Shaders Page One

From upper left going clockwise: Night Vision Scope, Lite-Bright, Cartoon Shading, Camera Output


Night Vision Scope

The night vision scope consists of three effects beyond the cheesy green tint night vision example.

• Minor random noise
• Lens distortion
• Blended circle edge

The random noise function generates a subtle graininess to the image. The random noise generator uses a single one line code from http://www.ozone3d.net/blogs/lab/20110427/glsl-random-generator/. It consists of the following:

• Dot product of the texture XY position with a vec2(a,b), where a/b values greater than 1.0 sharpens the image, less than 1.0 does an embossing effect, and if a == b == 1.0, has no effect.
• The sin() of the dot product is taken multiplied by c which generates random noise.
• By adjusting how large a and b are will control how much the random noise will cover details in the image.
• Example of this generator:

• fract(sin(dot(n.xy, vec2(12.9898, 78.233)))* 43758.5453);

The lens distortion is based on SynthEyes Lens Distortion Algorithm found on http://www.ssontech.com/content/lensalg.htm.

The blended circle edge surrounding the shader was based on Lighthouse3D's LED Shader Part 2. Where given the radius of the shader and the center position, a circular gradient is drawn around the region.


Lite-Bright



The Lite-Bright shader is based on Lighthouse3D LED Shader Part 1 and consists of five separate components:

• LED shader using a convolution kernel similar to CS 525 Week 4 lecture notes
• Shifting of every other row by half a pixel region
• Mapping the colors to the eight Lite-Bright colors (green, blue, red, yellow, orange, pink, purple and clear)
• Drawing a blended circle on top of the pixel region as a white highlight
  
• Drawing a blended circle on top of the pixel region to make the round peg shape
  

The convolution kernel from the LED shader divides the window into 'pixel regions' which effectively pixelates the window by sampling nine pixels in a 3x3 configuration to determine the average color of that region.

After the pixel regions are defined, I shift every other row by half a pixel region by using the following:

•  Determine what pixel region row we are in:
•  if( mod(pixelBin.y,2.0) == 0.0 )
•  Offset the x coordinate by half a pixel region:
•  TexCoord2 = vec2(TexCoord.x + oddRowOffset, TexCoord.y);
•  Then use those coordinates to define all pixel regions in that row:
•   pixelBin = floor(TexCoord2 / texCoordsStep);

Once the pixel region is defined I take the average color of that region and map the color to one of the Lite-Bright colors (green, blue, red, yellow, orange, pink, purple and clear). This was done by comparing each RGB values, combinations of the RGB, and averages of RG and GB.

The final two parts are similar to the blended circle edge used in the night vision scope, except the effect is applied to every pixel region instead of the whole shader window.

The first circle is used to create a slight shading around the center of the peg. The second circle is more like the night vision one and gives the pixel region a round peg shaped appearance. Also unlike the LED shader Part 2 example, the background the circle gradients to is not black, but rather a dimmed version of the pixel's color to give a subtle light effect around the peg.


Cartoon Shading

The toon shader is an expanded version of the webcam example. Here I used a technique similar to the Lite-Bright color mapping, but had more shades of red, green, blue, while, and a tan skintone pallet.

The bottom four images above are examples of what I used to determine the thresholds for many of the color pallets. The middle row is a lamp on my desk that changes through the colors of the rainbow over time. The bottom set consists of a Dunkin' Donuts cup, a Green Tea KitKat box, and a foam dart. Red, green, and blue were the easiest to isolate a calibrate. More composite colors like orange and cyan proved more difficult. The hardest colors to distinguish between was white and pale/tan skintones since they tend to merge easily. Especially when specular reflections occur.

Shaders Page Two

From upper left going clockwise: CRT scanline/Hologram, Lens Distortion with Chromatic Aberration, Thermal Imaging, Camera Output


Hologram / CRT Scanline

The hologram shader, more or less inspired from Star Wars, is the first one that was completed for this assignment. This shader is divided into three components:

• Luminance tinted blue
• Static scanlines
• Dynamic scanline
  

First I took the luminance of the image and increased the brightness a bit to get more of a glow effect. Next I tinted the image shades of blue based on a modulus of the y pixel position. Finally the moving white scan lines are based on a modulus of the y pixel and a timer from the main program.


Lens Distortion with Chromatic Aberration


This shader was based on the lens distortion used in the night vision shader as well as additional from http://devlog-martinsh.blogspot.com/2011/10/glsl-cubic-lens-distortion.html which also covers chromatic aberration.

The major difference between the night vision shader and this one is that the RGB values are separated individually by a refractive index, otherwise the code is very similar.


Thermal Imaging



This shader was inspired by just playing around with fragment shaders. This shader was generated using a combination of luminance thresholding and the random noise generated described under my night vision scope shader.

This shader uses high luminance to simulate the thermal image effect. Most images of this shader was used with a light shining directly on the subject. The desired effect was a result of modifying the luminance and noise generator parameters.

Shaders Page Three

Camera Bump Map (upper right) and Camera Output (Lower Left)


Camera Bump Map

This shader is based on the bump map examples from Andy's CS 525 Week 3 Lecture Notes. This shader attempts to use the webcam output, passes it through a luminance filter, and uses the resulting output as a normal map. The procedural bump map (upper left) is shown to compare the behavior between the procedural bump example and the Camera Bump Map shader.

For this page, the position of the mouse cursor controls the x,y position of the light source shining on both bump maps. Pressing 'r' will rotate the page around until '3' is pressed. This was the last shader developed, so there are no major refinements to the shader after it was deemed working.

Video Overview