This activity is modified from Liz Lerman's Critical Response Process.
We regularly conduct peer evaluations when working on our projects. The peer evaluations are in-progress evaluations that, if done correctly, will help you to build a better final project.
There are three primary roles in this process:
Artist: the person (or team) designing/building the program
Facilitator: the person responsible for keeping the process on track
Responder(s): Everyone else in the review group. It is the role of the responders to help give provide meaningful responses that will help the artist improve their work.
The peer evaluations follow a simple 4-step process:
Step 1: Statements of Meaning
Step 2: The Artist as Questioner
Step 3: Understanding the Process
Step 4: (Optional): Open discussion
All of this will be explained in more detail below
The goal of this process is to provide a scaffolded framework that helps facilitate a productive, non-judgmental conversation in order to improve your work.
The first step is to review the work you are about to evaluate. Since we predominately create programming projects, you will most likely review the most recent build of your classmate's programming project.
The goal of this step is to kickoff the conversation between the The Artist and the Responder(s). Since they both have a role to play in this step and, depending on the situation, you will be either the Artist or the Responder, let's break down each person's role individually.
In our peer evaluation work, when you are evaluating someone else's work, you are the Responder.
In this step, it is the job of the Responder try to create meaningful, non-judgmental statements about the work in order to start a bigger conversation about the project. We are NOT asking questions of the Artist nor are we looking for what is right/wrong or good/bad with the work. Instead, we are looking for elements we (as Responders) find meaningful, evocative, interesting, exciting, and/or striking in the work.
As you practice creating good meaningful statements, avoid using any of these words: why, good, bad, right, wrong, like, dislike. These words all carry either explicit or implied judgment.
Additionally, try to start all your statements with:
I notice(d) that . . .
If you can start your statement with "I notice that . . . " and avoid any of the words mentioned above, you should be able to craft a simple, non-judgmental "statement of meaning" about the work.
Creating a good "statement of meaning" takes some practice. Our gut reaction is to comment on whether we "like" or "dislike" something. In this class, when we evaluate work, we have to push our own personal feelings aside so we can focus on making more meaningful, constructive statements.
Let's try a little example. Review the program below. While you are reviewing the project, don't think about what you like/dislike. Instead, think about what you find meaningful, evocative, interesting, exciting, and/or striking about the work.
Stealth Infiltrator (GameLab)
If I was the Responder for this Peer Evaluation, I might create a statement like this:
I noticed that there was a splash screen when the game first loads rather than jumping straight into the game play.
So, what makes this a meaningful and/or constructive statement?
First, there is no judgment. It's just a simple neutral statement about a splash screen. It does not infer whether or not I think this is a good or bad use of a splash screen. Because whether or not I feel this is a good use of a splash screen is irrelevant (at least at this point in the process). I just make notice that a splash screen was used.
So, how is this helpful?
As a statement, it's not really helpful by itself. What it does, though, is it helps to start a conversation between the Responder and the Artist. That's how you should think of these statements, as conversation starters. The Responder makes a statement and then the Artist responds to that statement.
So, let's now take a look at the Artist's role in this first step.
In this first step, the Artists role is to respond to the statements made by the Responders. If the Responder makes a good statement, it gives the Artist the opportunity to respond in any direction they choose AND it doesn't put the Artist where she feels like she has to defend her position.
Take these two examples. The first is the statement I made in response to the Stealth Infiltrator game. The second is an alternate take on the same statement:
I noticed that there was a splash screen when the game first loads rather than jumping straight into the game play.
Why did you use a splash screen for you game. Why didn't you just let people get stated playing?
Do you notice any difference between these two choices?
Again, the first choice is an example of a meaningful, non-judgmental statement. The second is a question (rather than a statement). It also includes one of our words-to-avoid: "why."
Most people, when they hear the second option, are immediately put on the defensive because of the undertone of the "message." What the second option is really saying is, "I don't like what you did and I want you to justify to me why you chose to do it that way so I can then tell you that you're wrong."
This is not a good way to encourage the start of a conversation.
Let's go back to the first option:
I noticed that there was a splash screen when the game first loads rather than jumping straight into the game play.
This option actually encourages conversation because it allows the Artist to take the response in any direction.
Below I've listed a possible example of how the Artist might respond in this situation.
I provided the splash screen for two main reasons. First, I wanted new players to be able to access the CONTROLS screen and the OBJECTIVE screen BEFORE they start playing. Otherwise, they would jump in to the game and not necessarily know to control the player sprite or know how to score points. Second, was personal. I don't like games that start up right away when I click on them. I want to be able to click a PLAY/START button to start the game.
From here, if we chose, we could continue the conversation. The Responder now has more information to create follow up questions if he would like.
With distance learning, we are very unlikely to be able to have real-time conversations. Therefore, we have to manufacture some of these conversation starters and then provide a place where these conversations can continue.
Review the two programs below. As you do, look for what you find to be meaningful, evocative, interesting, exciting, and/or striking about the work. Then think of how you can create meaningful, non-judgmental conversation starters.
How Secure is Your Password? (AppLab)
Space Math (AppLab)
Now, use this form to submit some practice statements for each of these two programs. We will review these statements, as a class, at a later date.
The next step of the process shifts from the Responders making statements of meaning to the Artist asking questions of the Responder(s).
Remember, our peer evaluations happen while our project is in progress (not when the project is over). The goal of these peer evaluations is to improve the final build of your project.
So, in this step, if I'm the Artist, I am looking to create specific questions that will give me targeted feedback to help me improve my project.
Let's look again at the Space Math game from the last section. Let's say the original goal for this game was to help students in the primary grades improve their basic math skills. If I was the artist for that game, these are the types of questions I probably want to ask of my Responder(s).
Should I have a scoreboard that keeps track of stats (total questions, number correct, number incorrect, etc.)?
Should I include a timer in the game: maybe a time-limit per round or a time-limit for each answer?
Should I add some "motion" to the background? Would that make the game more interesting or more distracting from answering the math problems?
Should I eliminate the "punishment" (moving down a notch) when you enter an incorrect answer?
This step is a little hard to practice if you don't have a program YOU are creating. We will do our first practice with these types of questions when we get ready for the peer evaluation of our first project.
From a "learning programming" standpoint, this step is probably where you will grow the most (as a programmer). Again the Artist and the Responder each have a role in this step so let's look at each individually.
The rest of this section will be updated soon . . . before we do this step in the Peer Evaluation process.
TBD
TBD
TBD
This final step of the process is optional . . . and we will only really be able to do it when we are able to safely get back on campus.
When that time comes, I'll explain this last step.