Valid Tests

Evaluating our students’ learning is one of the most important tasks we undertake in our teaching, and it’s also one of the most difficult. Exams are a common tool for measuring student achievement, but they’re not easy to write.

When you give an exam, you’re gathering data on your students’ progress, so the test should be both valid and reliable. When we’re “testing what we want to be testing,” as Ory and Ryan put it, we start with a clear idea of what successful learning would look like—what should students know or be able to do by this point in the class?—and figure out how students can best give us evidence of that learning. If your test is not well aligned with your goals, “you will not be able to evaluate your students’ progress with any accuracy,” warn Clegg and Cashin.

It’s also important to think about how much time and priority you allotted to particular concepts so far this semester. The most important ideas deserve the bulk of our attention, student effort, and class time, so they merit a corresponding proportion of exams. As the research on the “testing effect” reminds us, if we test primarily on minutiae, our students may leave the course remembering those details, rather than the foundational concepts (Mastascusa, 2011).

Another important consideration is the proportion of each exam that tests higher-order thinking, as opposed to recall. Many test banks tend to rely on recognition questions, rather than analysis. If you want your students to be able to apply concepts in a variety of ways, you’ll need questions that demand that sort of thinking. Of course, your students will also need to practice this thinking in class, repeatedly: we can’t expect them to pick up new reasoning skills the day of the exam.

Tests and quizzes don't just measure learning; they should also be powerful learning opportunities. We’re asking students to retrieve and apply the ideas they need to take away from our courses, so ideally our exams are leading students through the thinking we want them to practice. They can keep learning by reviewing the test afterward, and reflecting on how they prepared for it. (For ideas, see our previous tip on exam wrappers.)

Practical guidelines and suggestions, plus a checklist for creating effective tests, are available here. CMU also offers great tips on designing effective exams.