Improving the Reliability of Your Rubric
Meet with teachers of similar courses to discuss the performance assessment/rubric.
Both should evaluate the students’ work using the rubric.
Use the Principles of Scoring Student Work Guidelines below for advice in scoring work
Discuss the scores and make decisions about conflicts that arise. Weigh criteria based on importance.
If two faculty members disagree significantly, possibly ask a third person to score the work.
If frequent disagreements arise about a particular item, the item may need to be refined or removed.
Principles of Scoring Student work
Know the rubric: Every score must be an attempt to apply the rubric’s language and meaning
Trust evidence, not intuition: Through common grading with peers, it requires us to base our judgments on the evidence we can see, not on what a particular person feels.
Match evidence to language in the rubric: Be sure you can justify your decisions with direct evidence from the student work
Weigh evidence carefully; base judgments on the preponderance of evidence: The score must be based on the overall performance as evidenced throughout the student work. It reflects what is generally true about the student’s overall performance within each of the scoring dimensions.
Know your biases; leave them at the door: Don’t let your biases trigger your first impressions that may color all judgments that follow.
Focus on what the student does, not on what the student does not do: It is easier to agree on what is than on what could be. Your score should be based on what is.
Isolate your judgment: One poor element does not equal a low-quality student work sample: Do not let one partial gap in the student’s work cloud the other aspects of the product. A student’s performance in one scoring dimension should not cloud your judgment on the scoring of other, unrelated dimensions.
Resist seduction: One good element does not equal a high-quality student work sample: This is known as the “halo effect”. The student submits one insightful and fluidly written paragraph, and after that he/she can do no wrong. One exceptional insight does not cancel out other aspects of the task that the student fails to address.
Stick to the rubric: Don’t try to assess what is not being measured in the rubric.