Redesigning Assessments for the Age of AI

(adapted from Derek Bruff)

1. Why does this assessment make sense for this course?

2. What are specific learning outcomes for this assessment?

3. How might students (ethically and unethically) use AI tools while working on this assessment?

4. How might AI undercut the goals of this assessment? How could you mitigate this?

5. How might AI enhance the assessment? 

6. Is there anything students can use GenAI to do for your class? For example:

   1. 1. 1. Invention/brainstorming (solving the “blank page” syndrome)

         2. Outlining

         3. Revision

         4. Research

7. Do the learning objectives support you using GenAI output as the starting point?

8. Can you intentionally infuse A.I. into an activity or assessment?

9. Can you build in a greater focus on the process (rather than product)?

10. Can you make the assessment more meaningful for students?

For a walk-through of how to apply some of these questions to an actual assignment, click here.

1. align your course learning outcomes to each activity and assessment so students know WHY you are asking them to do something in a certain way (this can increase student intrinsic motivation to learn rather than perform)

2. if you need to ban AI use on an assessment, then move that assessment to be proctored in-class. For more on securing assessments, click here.

3. replace some out-of-class assessments (like papers or projects) with in-class assessments like discussions and reflections, group case studies, presentations and debates (with Q&A)

4. make sure the assessment requires students to connect insights to in-class discussions and readings

5. revise grading rubrics to de-emphasize competencies that machines can do. Think about this - we used to include spelling, grammar and reference formatting in grading rubrics for essays and research papers, but we no longer do because machines can accomplish those goals. What other items do you have in your grading rubric that are now out-of-date because a machine can mimic achievement?

Sources: https://ecampusontario.pressbooks.pub/mcmasterteachgenerativeai/chapter/part-1-shorter-term-approaches/; https://www.umass.edu/ctl/how-do-i-redesign-assignments-and-assessments-ai-impacted-world 

1. move your assessments up Bloom's Taxonomy of Learning so that they focus on having students demonstrate their ability to apply, analyze, evaluate or create, instead of only remembering or understanding.

2. integrate AI into the assessments in ways that amplify rather than hinder student learning, such as allowing students to use AI for various research tasks, having students critique AI output, permitting students to engage with AI as a learning partner, asking students to research and select the most relevant AI tool to complete a particular assessment (https://www.umass.edu/ctl/how-do-i-redesign-assignments-and-assessments-ai-impacted-world) 

3. give students more choice and control in selecting how they want to demonstrate their knowledge (e.g., written product, oral presentation); providing choice and control can increase students' intrinsic motivations to learn

4. transition to more authentic assessments that enable students to apply their knowledge in practice, meaningful ways (e.g., instead of writing a research paper about a historical event, the students create a wikipedia page)

5. incorporate more "public accountability" for demonstrating learning, such as through team projects or peer reviews

For more ideas, see https://www.csu.edu.au/division/learning-teaching/assessments/assessment-and-artificial-intelligence/rethinking-assessments 

For a step-by-step guide in using a GenAI tool to redesign your assessments, click here.

If you are going to try to integrate AI into some of your assessments,  think about the following:

1. Sustainability: consider the environmental impacts that these GenAI tools have

2. Privacy: understand that not all students want to sign up for these tools because of concern with privacy; allow students to use tools that don't require them to log in and/or have alternative options for students who don't want to use the tools

3. Student Fears of Overreliance: students have a genuine worry about losing themselves and their ability to develop as a result of these tools, so consider teaching them how to use them properly

4. Bias: these tools amplify and normalize biases that exist in humans and in the history of human writing

5. Confabulations: these tools make stuff up; their output needs to be critiqued for accuracy

Also, when thinking about redesigning any assessments, it is always important to consider students with disabilities and avoid incorporating any changes that may negatively impact those students or require you to provide challenging accommodations.

For more on the ethical issues presented by Generative Artificial Intelligence, see this slide deck from Torrey Trust at U Mass Amherst