Any assessment strategy that relies on outpacing current AI limitations will soon become a solution that is out of date.
There is no “one size fits all” assessment type that will sidestep the challenges faced by genAI. Instead, we recommend deploying a variety of assessment types to evaluate student understanding.
Click on a strategy below to jump to more details.
Closed book, in-person exams
Live presentations
In-class assignments
Low stakes assessments of all kinds
Checking your questions against AI responses
Developing authentic assessments
Evaluating process & revisions
Heavily weighted “high-stakes” exams
Over emphasis of recall and basic facts
Simple, generic essays
In-person exams that exclude the use of electronic devices will not need to change as students cannot use AI tools to take them. However, students might use generative AI as a study partner to prepare for an in-person exam in the same way they might study with a classmate.
Relying exclusively or excessively on hand-written and oral assessments can result in inequities for non-native English speakers, students with disabilities requiring accommodations, or neurodivergent learners from fully demonstrating their learning.
To ensure you are providing an environment where everyone has an opportunity to demonstrate their capabilities, the Moses Center recommends addressing your assessment plan with students as early as possible. Provide information on your assessment style in your syllabus, so that students who may need additional accommodations can request them as soon as possible.
The delivery of a live presentation is less impacted by genAI. Live presentations require students to have planned and prepared a talk to communicate their key message. In preparation for live presentations, students may use generative tools to brainstorm, summarize resources, or generate outlines. However, the presentation and Q&A itself require students to respond in the moment, drawing on their preparation and prior knowledge. When students are speaking live about course content and taking questions from other students or faculty, they are practicing the kinds of conversations an industry expert may have in the field and by doing so, learning along the way.
Some students may find this form of assessment particularly challenging (e.g., students with disabilities, non-native speakers, those who struggle with public speaking). Consider diversifying the types of assessments you use.
Live class discussion and other graded activities can be an engaging way for students to make arguments, provide supporting evidence and respond to counterclaims in real time. For faculty, they are a way to assess students’ levels of proficiency in a skill or comprehension of content areas in the moment.
Examples may include:
Socratic discussions
Warm or cold calling students
Debates
Negotiations
Case study analysis & discussion
Students might want to use generative tools to prepare for these discussions. Providing students with a rubric or other guidance on how their contributions will be evaluated can help them focus on the right source material rather than over-relying on generative AI for preparation.
Formative assessments are a great way to identify where students are struggling in their learning so you can make adjustments in your instruction. The best are frequent, low-stakes tasks or assignments, graded for completion or ungraded, so students have little incentive to rely on AI.
Some examples of formative assessment include:
Practice problems (partially solved, or with full solutions provided)
Short quizzes (with immediate feedback)
Small group problem solving live in class
Exit tickets / weekly memos
Required discussion forum posts before or after class
Run your questions through generative AI, preferably using the more powerful models that are widely available, such as GPT-4. Get an idea of where the tool performs well and where it is limited or superficial in its responses. Consider how likely it would be for a student to generate a “good”, “average”, or “failing” response based on your prompts/questions.
After you've tested how well generative AI performs on your assessments, you may find you want to revise some of them. Consider using the authentic assessment framework, which asks students to apply skills and knowledge to come up with original solutions in real-world contexts.
Authentic assessments include:
Real world problems
E.g., students are asked to evaluate a real life firm for investment opportunities.
Authentic product and and audience
E.g., students research their assigned firm and present an investment analysis with supporting data to an “investment committee” of other students in the class and/or outside experts.
Reflection and feedback
E.g., students share internal feedback throughout the assignment with their team members throughout the project, and also receive feedback from the “investment committee” upon submission.
Stern faculty have submitted more examples of Authentic Assessment, which you can find by clicking the button below.
Use a pedagogical approach called “scaffolded assessments” to break down assignments into smaller parts that build upon each other, leading to the completion of a larger final product. This approach lowers the stakes of any one deliverable and makes success on the final product more likely. If you ask students to compose all components in the same shared document, you will be able to review the composition and revision process.
Larger assignments divided into stages can take many forms.
Outline
Content draft
Design draft
Final presentation
Feedback
Topic
Outline
Draft
Final
Feedback
Market research
Problem identification
Possible solutions
Final recommendation
Thesis
Topic Sentence
Literature review
Proposal
If you permit or encourage use of generative AI tools at some stages of the process, e.g., brainstorming, copy-editing, and planning, you can add a reflection component in which students reflect on the contribution of genAI to their learning process. This reflection may involve considering why they chose to use genAI at certain points, assessing the positive or negative impact it had on their work and learning. Encouraging students to critically analyze their use of genAI fosters metacognitive skills and helps them develop a deeper understanding of the tool's benefits and limitations.
Adoption of these practices may take time. Need examples or assistance with developing authentic or scaffolded assessments? The Learning Science Lab is eager to work with you.
While these types of exams and assignments can be convenient to employ, they are the most susceptible to plagiarism and other kinds of academic dishonesty. High-stakes assessments can create significant stress and anxiety for students and shift their focus from fostering a deeper understanding of the subject matter to the grade they will receive.
We recommend
employing diverse assessment types across the course,
providing students with multiple opportunities for feedback, and
incentivizing the knowledge construction process.
A balanced assessment approach provides a more comprehensive view of students' progress and encourages a deeper and more meaningful understanding of the subject matter.
Including too many simple knowledge recall questions in an online or take-home exam will make your assessment more susceptible to AI generated responses.
Instructors should move away from including recall questions in exams and instead incorporate them throughout the course as frequent, low-pressure, formative assessments that allow students to check their understanding. In general, employing a range of assessment types during your course will encourage critical thinking, discourage cheating, and allow students to demonstrate more of what they know.
The lowest tier of Bloom's taxonomy asks students to recall or recognize basic facts. Questions that do this can be useful, but are more susceptible to AI-generated responses. While Bloom’s Taxonomy, is an imperfect representation of the knowledge construction process, since it excludes the social process of learning, it does offer a simple framework that faculty can use to map activities to different cognitive activities.
ChatGPT and other genAI tools can produce seemingly convincing answers to common essay questions, however we do not advise eliminating essays and final papers from your assessment toolkit. Instead, consider the following strategies for improving essay assignments.
Use a rubric to clearly define the learning goals and criteria for evaluation.
Scaffold the assignment so that students can complete smaller tasks like topic approval, outline review, draft review, final submission, etc.
Incorporate both traditional essay writing and genAI tools as complementary components of the writing process.
Require a minimum number of citations of relevant course topics and readings.
Offer students some autonomy in selecting their essay topics or final paper themes as it aligns with the overall learning goals. Allowing them to choose subjects that align with their personal interests, experiences, or career goals can increase engagement and ownership of the assignment.
Use a problem-based approach: Frame the essay as a response to a real-world problem or challenge, encouraging students to think critically and develop practical solutions, rather than just summarize concepts, theories, or facts.
Derek Bruff, formerly of the Vanderbilt Center for Teaching, writes at his blog Agile Teaching, about how he redesigned an essay assignment by answering a series of six critical questions to help craft a more compelling assessment, all while feeding the revised prompts to ChatGPT along the way.