Lab 5: Image Manipulation

Learning Goals

In this lab, you will manipulate and transform images. A key focus is to give you lots of practice with using loops, as well as seeing the practical implications of working with mutable data structures. Your learning goals are the following:

• Import and export picture files in PIL

• Use your programming knowledge to manipulate pixels in picture files

• Implement algorithms to transform pictures

• Understanding how mutable data structures behave

• Dream up new creative image manipulations

Setting Up Your Repo

This assignment consists of both individual and pair programming exercises. As in previous assignments, you will create only one private repository between the two of you. For this lab, you will be working directly with your pair partner, creating code together. This is possible in Repl.it by using a Shared Repl.

Just like in Lab 3, if you visit the page https://github.com/spis2022, you should find a repo already set up for you and your pair partner (or trio) with the name lab05-name1-name2 (or for a trio, lab05-name1-name2-name3). The name should match your pair (or trio) name from the pair-partner spreadsheet. There is no starter code in this lab. Import this repo into a Shared Repl.

What comes next is the individual portion of the lab. The goal is for each of you to get familiar with the PIL library and how to work with images, before moving on to the pair programming portion of the assignment.

Note: Two students simultaneously writing code in a Shared Repl (without running it) seems to work fine on Repl.it, but trying to run code in a Shared Repl while your partner is working might be momentarily disruptive. You will want to give your partner a heads-up before running your code.

Individual Portion: Getting Familiar with PIL

This first part of the lab is individual. However, we want you to work synchronously with your partner and support them as they go through these exercises. When you created the shared repl, it created a file called main.py by default, which you will use for the pair programming portion of this lab. For this individual portion, however, each pair partner will create and work in a file called lab05Warmup_Name.py (where Name is the first name of the pair partner). For example, if the students are named Felix and Ryan, Felix should create and work in a file called lab05Warmup_Felix.py, and Ryan should create and work in a file called lab05Warmup_Ryan.py. You can create a new file by going to the "Files" tab in the left sidebar, clicking the "Add file" button (icon is a piece of paper with a plus sign), and entering a file name.

Normally, in Python, you can run any .py file, but unfortunately, Repl.it will only run the main.py file. As a workaround, add the following to the top of main.py (where Felix and Ryan should be replaced with you and your partner's names):

print("Running lab05Warmup_Felix.py") # let us know it's running lab05Warmup_Felix.py

import lab05Warmup_Felix # this will cause lab05Warmup_Felix.py to run

print("Running lab05Warmup_Ryan.py") # let us know it's running lab05Warmup_Ryan.py

import lab05Warmup_Ryan # this will cause lab05Warmup_Ryan.py to run

Before you get started with programming, we need to point out one important fact. The goal of the lab is to practice coding, and specifically the use of (nested) loops and the intricacies of data referencing. For many of the tasks we ask you to do, there already exists a function in the PIL library that implements it. However, it is important you re-implement the functionality from ‘first principles’ so you get to practice and hone your programming skills. So implement the functions as we ask, don’t simply notice that a similar function already exists somewhere in a Python library.

Getting Familiar with PIL

In this lab, we’ll work with the Python Imaging Library (PIL) which is a graphics library, like turle, but designed for working with image files. Download the picture below and store it in your GitHub repo working directory for this lab (the one you cloned above). You can do this by right clicking on the image and selecting the option to save. Be sure to save the image as bear.png. Then, in your shared repl, upload bear.png by going to the "Files" tab in the left sidebar, clicking the menu (3 dots) button, clicking "Upload File", and selecting bear.png from wherever you saved it. You only need to do this once for your pair.

Then, once PIL is installed, in your individual lab05Warmup_Name.py file, write the following code:

from PIL import Image

bear = Image.open( "bear.png" )

The first line instructs Python to import the Image portion of the PIL image library. Whenever you use a function from this library, it will start with Image., indicating you are looking for that function in this specific library. The next line of code opens an image and stores a reference to it in the bear variable.

Saving Images as Files

As you call various functions on a PIL Image object to modify the image it stores, the changes are only stored in local memory. You are not actually changing the bear.png file, and any of the changes you make will not be saved after the program finishes running. The reason is that the Python functions you’ve used are not directly processing the stone bear image file itself: instead, when you open an image, PIL makes a duplicate of that image and loads that duplicate copy into the computer's RAM. This technique of modifying a loaded copy of a file in memory (which is relatively fast) rather than directly handing a file on the disk (slow) is very common.

Normally, you can use PIL to view the current image (with all modifications you've made) by calling the show() function of the Image variable (e.g. here we would call bear.show() to view the image stored in the bear variable), but unfortunately, Repl.it doesn't currently support this. However, it does allow you to click on an image file in the "Files" tab of the left sidebar to view it, so we will use this functionality as a work-around. Specifically, any time you want to view the image stored in the bear variable, you can use the save() of the Image variable to save the current image into a temporary file, which you can then click on in the "Files" tab of Repl.it to view. Any time you want to save the current image into a temporary file, run the following line of code, which will replace the file if one exists (replace Name with your name):

bear.save("tmp_Name.png") # create/overwrite tmp_Name.png with current image

As mentioned, running this exact command multiple times in a row will result in replacing the same file over and over again, so if you want to run this command twice in a single execution of your code, you may overwrite the first image file before you can view it. Either call this just once at a time, or if you wish, you can specify different image filenames each time (e.g. "tmp_Name_1.png", "tmp_Name_2.png", etc. to avoid overwriting).

In order to view the images, you will first need to upload tmp_Name.png (this can be a duplicated bear.png that you rename tmp_Name.png) to your Repl.it. After you run bear.save("tmp_Name.png"), you can see the modified images.

Basic PIL Functions

Before you move on, add comments to your file to explain each line of code.

How many pixels is your image composed of? The following command could be helpful:

bear.size

Next, we are going to access a specific pixel from the image by using the getpixel() function. This function is called on an Image object, and the arguments of this function are the desired pixel's x coordinate and y coordinate. The function returns the resulting pixel object. This pixel is a tuple representing the RGB (Red, Green, Blue) values of the pixel (in the case below, at location x = 100 and y = 200).

pixel = bear.getpixel( ( 100, 200) )

Note that, in the image grid, the axes are a little different with respect to the usual 2D Cartesian axis, in that the axes count from upper-left to bottom-right. For example, in the following 18 x 18 image grid, and the coordinate (11, 7) is the grey block. Note that the index starts at 0, meaning the top-left block is at coordinate (0, 0).

You can check the value of the pixel and verify it is a tuple of RGB values (3 numbers, one each for R, G, and B) by adding this line of code to your Python file:

print(pixel)

Now that we have accessed a pixel, let’s see how to modify the colors of individual pixels in the image. To modify a pixel, use the putpixel() function:

bear.putpixel( (100, 200), (0, 0, 0) )

The putpixel() function modifies one pixel at a time and takes two arguments: (1) a pixel coordinate represented by an (x, y) tuple, in this case (100, 200); and (2) a tuple representing the RGB color to set the pixel to, in this case color (0, 0, 0), which is black.

Add the putpixel() function call above to your code and view the bear image. Can you find the modified pixel at (100, 200)? If you have a hard time seeing the modified pixel, try the following code instead to turn a range of pixels black. Make sure you understand what this code is doing.

for i in range(100):

bear.putpixel( (i, 200) , (0, 0, 0) )

For detailed information on the functions we have used so far and the PIL library in general, you can read the documentation for the Image module.

Inverting the Colors

Now, let’s try to modify our image in an interesting way. Way back in the days of film cameras and chemical processing of photo images, one step in the processing produced a negative image. We can achieve the negative (aka inverted) effect digitally by subtracting each of the original RGB values of a pixel from 255. For example, if the pixel RGB values are (34, 67, 87), the new RGB values of that same pixel should be (221, 188, 168): basically 255-34, 255-67, and 255-87. Of course, you need to do this not just for one pixel, but for all the pixels in the image.

Create a function called invert as shown below, which implements this operation for all pixels of an image. Also delete any code you no longer need. Note that the function below is incomplete. It is up to you to fill in the missing lines of codes.

def invert( im ):

''' Invert the colors in the input image, im '''

# Find the dimensions of the image

(width, height) = im.size

# Loop over the entire image

for x in range( width ):

for y in range( height ):

(red, green, blue) = im.getpixel((x, y))

# Complete this function by adding your lines of code here.

# You need to calculate the new pixel values and then to change them

# in the image using putpixel()

Now add the required code to import the PIL library, open the bear.png image, and call the invert() function on the variable that points to this image.

invert(bear)

Basically, the code above calls the new function on a specific Image object (in this case, referenced by the variable bear). Finally, don’t forget to add a line of code to actually save and view the new image (remember to include your name in the filename to avoid accidentally overwriting your partner's saved image). When you now run the code, your result should look similar to this.

For this exercise and for all subsequent ones, make sure you test your code on the bear image and on at least one other image. We suggest using your own picture available in one your earlier GitHub repos :-)

Modifying Only a Part of the Image

Next, create a new function in your file:

def invert_block( im ):

Its functionality should be the same as that of invert(), except that it only inverts the pixels that are in the upper-right quadrant of the image (so it only inverts 25% of the image) and leaves the others unchanged.

To test if it works, apply invert_block() to the bear image.

Now, what happens if you apply invert() first and then next apply invert_block() to the same image? Make sure you understand why this is happening. Confer with your partner.

Submit Your Code

As you and your partner finish this individual section of this lab, submit your code by pushing it to GitHub. Be sure to check your GitHub repo in your web browser to verify that your code successfully pushed.

At this point, you are done with the individual portion of this lab. If your partner is not yet done, DO NOT CONTINUE; instead, have fun with creating new image transformations of your own. For example, what happens if you swap color channels (e.g. R becomes G, G becomes B, B becomes R)? What happens when you delete (i.e., set to 0) one or more of the color channels? Can you modify your code such that these transformations only apply to every other pixel rather than every pixel or to a specific area in your image? Check out some of the things that are possible at the end of this assignment under Creative Challenges, including green screen manipulations. You could be busy for many hours... However, once your partner finishes the individual portion, sync up again and start the pair programming portion of the lab. At the end, get back to creating new artistic image manipulations...

Pair Programming Portion: More Advanced Image Manipulations

STOP!!! Do not start on this part of the lab until BOTH partners have completed the individual portion. When you do start, make sure you use pair programming!

First, select the main.py file from the "Files" tab on the left sidebar of Repl.it. You will place all the code of the pair programming exercises in this file. Comment out the lines of code you wrote previously that executed each of your Individual Portion code files (by adding a # symbol at the beginning of each line) so they are no longer executed, e.g.:

#print("Running lab05Warmup_Felix.py")

#import lab05Warmup_Felix

#print("Running lab05Warmup_Ryan.py")

#import lab05Warmup_Ryan

Make sure you add appropriate comments, including a header comment. For each exercise, come up with a solution outline by discussing with your partner. Don’t be in a hurry to start coding unless you have come to a fairly clear idea of a solution strategy and your first steps. Also, push your code to GitHub regularly! This gives you an online copy to guard against accidentally deleting your work. Also, it is a good habit to get into.

Reducing the Color Space

Now that we have some experience changing the colors in a picture, we will continue with some more color manipulation examples that manipulate each pixel individually.

Grayscale

To create grayscale images, we make use of the concept of image luminance. In layman's terms, luminance is how bright or dark the colors in a pixel are (compared to white). As Wikipedia calculates it, luminance is 21% red, 72% green, and 7% blue. Intuitively, this makes sense because if you think of standard red, green, and blue, green is the lightest and thus has highest positive impact luminance, while blue is darker and has a lower value for luminance.

Write a new function called grayscale() that takes an image as a parameter and modifies it to make it grayscale. For this, you'll want to do something similar to invert(), except that we will first calculate the luminance of a pixel and then set each of the three color channels to this value. Since luminance is an indication of how white or black a pixel is, just insert the same value in each of the three color channels.

Hint: Getting an OverflowError: unsigned byte integer is greater than maximum? This might be because your luminance calculation results in RGB values higher than 255. Make sure that all of your percentages add up to 1. Also, if you get integer argument expected, got float, it may mean you are trying to assign red, green or blue a floating point value (rather than an integer). You may solve this by using a typecast, c = int(a/b), or doing an integer division, c = a//b, versus the floating point one, c = a/b (See also integer division).

Binarize

Now, write a function called binarize(im, thresh, startx, starty, endx, endy) which modifies a portion of im to be black and white based on a threshold luminance value (thresh) specified by the user. This threshold is a brightness value between 0 and 255: if a pixel’s luminance (see earlier) is greater than the threshold value, then it should turn white, and if it is less than the threshold value, then it should turn black. This function should only apply this "binarize" operation to pixels that are inside a box, where startx and starty represent the x and y coordinates of the upper-left corner of the box, and endx and endy represent the x and y coordinates of the lower-right corner of the box. Your function should check that these numbers are valid (make sure your checks are complete!). If they are not, the function should not modify the image, but should print a warning instead.

Did you push your code to GitHub? It is a good idea to do this regularly.

Geometric Transformations

The following functions take an image as an argument and do some geometric transformations on it.

Vertical Mirroring

Write mirrorVert(): This function takes an image and modifies the image to mirror the photo across its horizontal axis (i.e., so that the top part is mirrored upside-down on the bottom of the image). Hint: Think carefully about which pixels you need to loop over and where each pixel in the top half needs to be copied to create the mirror effect. Start with concrete examples. Then, derive the general formula based on the pixel’s location (x, y) and the height and width of the image.

Horizontal Mirroring

Write mirrorHoriz(): Same as above, but mirroring across the vertical axis. Hint: Instead of replacing the bottom rows with the reversed top rows (as you did in mirrorVert()), you'll replace the last half of the pixels in every row with the reversed first half of the pixels.

Vertical Flipping

Write flipVert(), a function which flips the image in a picture along its horizontal axis (so the result is that the bottom is on the top and the top is on the bottom). Again, think carefully about where each pixel needs to end up, how far your loop needs to run, and be careful not to overwrite the pixels in the bottom half of your image before you’ve copied them over into the top!

When was the last time you pushed your code to GitHub?

Geometric Transformations Returning a Copy of the Image

The next three functions operate differently with respect to what we have done thus far. Instead of manipulating the original image, they will create a modified copy of the image and return it (i.e., use the return statement). They should NOT modify the original image.

The command below can be helpful. It creates a new image im, as a color image (this is what the RGB means), of a certain width and height given by the tuple:

im = Image.new('RGB', (width,height))

Scale

Function scale() takes an image as a parameter and creates a copy of that image that is scaled to be half its original size. Then, it returns this scaled copy (so you are going to have a return statement now). Hint: one way to do this is to skip every other pixel when copying from one image to the other. Be careful with your coordinates so that you do not go out of bounds in the smaller image.

Blur

Function blur() also returns a modified copy of the image. This copy will be a blur of the original image, created by combining neighboring pixels in some way (entirely up to you). You might consider averaging the RGB values of a designated "square" of pixels, then changing each of these pixels' values to the average.

... GitHub!! ...

Random Grid (Challenge Problem)

Function randomGrid() also returns a copy of the original image. To create this copy, it divides the image into an n x n grid (where the n is up to you, or you can make it an argument of the function), and it randomly sorts the pieces of the grid across the image, "sliding puzzle"-style. Hint: you can use Python's random library (just Google this).

import random

Submit Your Code

Submit the pair programming portion of your code, which should all be in the main.py file, to GitHub using the usual approach.

Challenge Problem 1: Hiding Images in Plain Sight

Steganography is a way to hide a secret message inside an ordinary file. There is a great article on BBC.com about this idea and how criminals may be using it.

For this challenge, you will hide a secret black-and-white image inside a standard image. The idea is to pick a color channel (R, G or B; the choice is yours) in the standard image. Let’s assume you chose to work with R. Each pixel has a value for R, which is between 0 and 255. The reason behind this range is that it is represented by an 8-bit number: with 8 bits, we can represent the non-negative numbers from binary 00000000 (0 in decimal) to 11111111 (255 in decimal). Now, imagine we have a pixel with an R value of binary 01011101 (decimal 93). Would you be able to tell the difference if I change that value to 01011100 (decimal 92)? Probably not. This is what we will use to hide our secret black-and-white image: we will take the R value of each pixel in the standard image and replace its least significant bit (i.e., the right-most bit if I were to write the value in binary) to either 0 or 1 based on whether the corresponding image in the secret message is black or white.

Basically, we are hiding the black-and-white image in the least significant bit of one of our color channels. To get the hidden message back, you simply need to check the least significant bit. A value of 0 means the hidden message has a black pixel there, while a value of 1 means it has a white pixel. Also note that having a least significant bit (LSB) of 0 means the number is even, and having an LSB of 1 means the number is odd.

For the first part of this challenge, download the image below (hiddenbear.png). It has a secret message embedded inside its red color channel in the manner described above. Decode this message!

For the second part, write your own function to hide a secret message. Create a black-and-white image and hide it inside another image! You can extend your code to hide three images, one in each color channel.

Challenge Problem 2 : Time to Get Creative!

Now it’s time to create your own effects! You can do them individually, as a pair, or with another partner if you wish (in which case, just select one of your repos and put in a header comment specifying who worked on this). Please be sure to include a comment or note to the tutors explaining what you did. There is literally no end to this assignment. Below, you can find some examples of what you can do. All of this was created with Python code, not Photoshop! :-)

A really cool technique is the green screen (this is how the last image below was created). This lets you superimpose yourself onto any other image. Ever wanted a picture of yourself on Mt. Everest or on the moon? This is the way to do it! The way the green screen substitution works is that you need to identify all pixels that correspond to a range of green, and then replace them with pixels from another image. If you have a green screen (or really, any large piece of cloth that is a single solid color; doesn't have to be green), try it out! Have fun!