Effective Problems

How do you write effective problems?

Writing problems for homework and exams is my favorite part of teaching when I am not directly interacting with students. I think of it as an art form. We are trying to construct problems that lead students to do particular kinds of mathematical thinking. A good problem does more than check whether students can reproduce a technique: it gives them an opportunity to discover, connect, explain, or wrestle with an idea. Not every problem has to be a long adventure, but ideally, students leave each problem understanding, and ultimately remembering, what they were supposed to take away from it.

When I first started writing problems, I was intimidated by the prospect of writing good ones. Over time, I have developed a collection of practices that have helped me, drawing from my own experience as well as advice from mentors and conversations with peers. 

Here is a list of best practices:

1. Ask yourself questions throughout the writing process. What mathematical thinking do I hope students will leave the problem having done? What skills do I want students to practice or demonstrate while solving this problem? Is it clear what question I am asking students to answer? Are there unintended solution paths that are much easier or harder than I intended? Could this problem be more tedious than anticipated? 

2. Don't assume that students share your cultural or experiential background. Consider whether the topics of your word problems or explanations rely on knowledge that may not be culturally universal. A washer is a good example. At the blue-collar high school I attended, everyone knew what a washer was, so my teachers could ask students to find the area of a washer without defining the shape first. At Cornell, where students come from much more varied backgrounds, a washer is not necessarily a familiar point of reference. 

3. Make the arithmetic as easy or as hard as you intend it to be. The arithmetic of a problem should be challenging only if you intend the arithmetic itself to be part of the challenge. If the goal is to test whether students understand a mathematical concept, don't accidentally make the problem harder by giving them unnecessarily complicated arithmetic.

4. Ask students to explain their reasoning. In my opinion, being able to explain mathematical reasoning is one of the most important things that mathematics education gives students. Asking students to explain their reasoning also helps us as graders understand where their problem-solving process went wrong, allowing us to give more useful feedback than we could from an incorrect answer alone.

5. Make the language serve the mathematics. A problem should not be difficult to parse unless parsing the problem is part of the challenge you want the students to face. Read your problems as though you were a student encountering the material for the first time. If a sentence can be simplified without changing the meaning mathematicaly, consider simplifying it. 

6. Give students both straightforward practice and opportunities to put ideas together. Homework should include problems that help students understand and practice basic concepts, as well as problems that combine multiple concepts or require more complicated reasoning. Students need opportunities to develop fluency with individual ideas before they can be expected to use those ideas together in unfamiliar or tricky situations.

7. Exam problems should feel familiar. Exam problems should be very similar to problems students have already done. Especially in upper-division classes, this does not mean simply changing the numbers on an existing problem. Rather, taking into account the mathematical maturity of your students, write problems that make them think, “I've basically done this type of problem before.” When I asked around the math department at Cornell for advice on problem writing, I heard the same thing multiple times: “If you are excited by a problem, it is probably too hard for an exam.”

8. Don't make students' answers to one part determine whether they can demonstrate what they know in another. Multipart questions in which the answer to a later part depends on the answer to an earlier part can be a grading headache, but more importantly, they can prevent students from demonstrating understanding that they do have. 

9. Write silly problems. Math can be fun. At my undergraduate college, Harvey Mudd College, everyone took a freshman special relativity class, and most of the homework and exam questions were written about Robespierre the Radical Rhino. That was essentially the only way the problems were made “fun,” and yet we all loved it. The silliness made the problems more memorable and actively engaged us in solving them.

If you want to make a problem more challenging: 

1. Have students determine which tools to use. Instead of telling students what method to use, give them a situation where they have to decide which tools or theorems are appropriate. This is probably one of the easiest ways to increase difficulty. For example, instead of “Use the Intermediate Value Theorem to show…,” ask students to show the result and let them decide how. 

2. Ask students to determine whether a claim is true. “Prove that…” tells students that the statement is true. “Is the following statement always true?” requires them to decide whether to prove it or find a counterexample. My undergraduate advisor used to write all of his exam questions as “prove or disprove” because he was afraid of accidentally introducing a typo that made a true statement false. It was not until a student pointed this out that he realized he had inadvertently made his exams substantially harder: students had a much larger search space for finding a valid approach than they would have if he had simply said “prove.” 

3. Give students information they don't need. This challenges students to identify which information is relevant to the problem and which is not. On a calculus homework, I once wrote a question about draining a bathtub and mentioned that the bathtub was pink. Several students told me afterward that the superfluous information had genuinely tripped them up. This is a useful technique, but use it carefully: irrelevant information should challenge students to distinguish what matters, not simply confuse them.

4. Give students an incorrect solution and ask them to find the errors. This asks students to critically read someone else's mathematical reasoning, identify where it goes wrong, and explain why.

5. Make the solution slightly uglier than usual. Students can sometimes become accustomed to thinking that an answer must be correct because “everything cancels out nicely.” A solution that does not simplify quite so beautifully can force students to check their reasoning rather than relying on the appearance of the final answer. I have found this especially effective in calculus classes.

6. Introduce a new definition in the problem. This is especially useful in proof courses, where one of the goals is to help students become comfortable proving properties of newly introduced mathematical objects. Giving students a definition they have never seen before and asking them to reason from it is good practice for the kind of mathematical thinking that we are trying to foster in a proofs course.

If you want the problem to be less challenging:

1. Provide more scaffolding. Break the problem into larger steps or provide some of the solution structure. Another option is to provide the answer and ask students to explain how you get to it. These approaches can reduce the number of decisions students have to make while still asking them to engage with the underlying mathematics.

2. Tell students what method to use. If the challenge you want is executing a method rather than choosing one, say so. “Use the Intermediate Value Theorem to…” is substantially easier than “Show that…” where students have to figure out what theorem might apply. 

3. Make important information visually obvious. If the problem is a word problem, consider presenting the important information separately from the surrounding prose. Bold key quantities, use bullet points, or otherwise make the structure of the problem visually apparent. This can reduce the reading and information-management demands without changing the mathematics.

4. Make the numbers smaller. If arithmetic is not what you want students to struggle with, use numbers that are easy to work with. Often, when students are learning new mathematics, even arithmetic they feel comfortable with can get in the way of the point of the problem. I remember when I was first learning integration: I started making mistakes in basic addition and subtraction because I was so focused on the new mathematics. 

5. Give students a diagram or representation. If students would ordinarily have to construct a picture, table, graph, or other representation for a problem, providing it can remove a substantial hurdle, as long as the representation itself isn't what you're trying to assess. 

If you have thoughts on writing effective problems that aren't here, or if you disagree with anything I've written, please email me (ab2439@cornell.edu)! Many of the ideas on this list came from conversations with other educators, mentors, and colleagues. My goal is for it to continue growing from the experiences and perspectives of many different educators.

Example Problems