Rethinking Assessment

The conversation around AI and assessment tends to move quickly toward rules: What should students be allowed to use? What should teachers prohibit? How much AI is too much? While these questions matter, they may not always be the best place to begin. A more useful question might be: What are we actually trying to learn about a student when we give an assignment? This edition of the AI Chronicles again explores AI and assessment, providing practical solutions to daily classroom needs and larger reflections on the nature of assessment itself.

To start, sometimes we want to know whether a student can write independently. Sometimes we care more about the quality of the final argument than how it was produced. Sometimes the process matters as much as the product. And sometimes the skill we are trying to teach may eventually include knowing how to work thoughtfully with AI.

Since those are different purposes, they require different approaches to assessment. 

There Is More Than One Philosophy of AI Use

Educational discourse sometimes talks about AI as though educators must choose between two inflexible positions: embrace it or ban it. The reality, however, is much more interesting. Teachers are already developing a range of approaches depending on the age of their students, the subject they teach, and the particular skills they are trying to assess. Add to that a series of high stakes news stories surrounding AI and policy—such as NYC's ban on AI in all classrooms before 9th grade—and the field becomes that much more interesting to consider.

According to Mayor Mamdani,

Children need teachers and human connection in order to learn and grow. They need to develop skills alongside their peers, build relationships with educators and wrestle with tough problems on their own... The tech industry wants us to believe that A.I.-powered early education is not only inevitable, but necessary. We do not see it that way. That’s why we’re implementing a moratorium on generative AI for students in 2-K through 8th grade and spending the next year studying the impacts of this technology.

Regardless of your personal stance and pedagogical approach, I would encourage you to clearly convey these expectations to students. To help facilitate this practice, I have put together a "stop light" graphic to helping students understand your expectations surrounding AI (see above).

The AI-Free Approach

If the purpose of an assessment is to determine whether a student can independently analyze a passage, construct an argument, solve a problem, recall information, or write fluently, removing AI is entirely appropriate. 

In these situations, limiting technology is both a rejection of AI and a way of protecting the validity of the assessment.

Possible examples:

• timed writing

• handwritten responses

• Socratic seminars

• oral examinations

• live problem solving

• reading annotations completed in class

• closed-device quizzes

• individual conferences

The AI-Resistant Approach

While the tech sector would have us believe that there is probably no such thing as an entirely "AI-proof" assignment, I am much more skeptical. For starters, assignments that make outsourcing the thinking considerably less useful are becoming considerably more apparent across pedagogical practice.

AI-resistant assessments often depend on classroom experiences, conversations, local information, individual choices, accumulated work/process analy sis, or material that an AI system cannot easily reproduce without the student's participation.

The point shouldn't be to trick students or catch them using AI; instead, I would encourage you to design assessments in which the student's presence cannot be replaced.

Some possibilities

Build from classroom material. Ask students to respond to a discussion, experiment, demonstration, seminar, field experience, or collection of texts that developed inside the course.

Make process visible. Collect notes, proposals, drafts, annotations, revisions, conferences, and reflections rather than evaluating only the finished product.

Add an oral component. A five-minute conversation about a paper can reveal an extraordinary amount about what a student understands.

Use local material. Community issues, school data, local history, interviews, observations, and place-based questions give students something more specific to work with than information available across the internet.

Ask students to make (and defend!) consequential choices. Instead of simply producing an answer, require students to explain why they selected particular evidence, rejected another approach, changed their minds, or made a particular decision.

Return to the work later. Ask students to revisit an earlier argument several weeks later, defend it, revise it, or challenge it using something they have learned since.

The Bounded AI Approach

Another possibility is to allow AI for particular parts of an assignment while protecting others.

A student might use AI to brainstorm questions but not draft the response. Or use it after writing to receive feedback. Or compare an AI-generated interpretation with their own.

This moves the conversation away from the binary language of "AI allowed" and "AI prohibited." Instead, the teacher defines where AI belongs in the learning process and why.

For example:

• AI may critique a completed draft, but students must decide which suggestions to accept.

• AI may generate several possible solutions that students then test.

• Students may use AI but must document and discuss how it influenced the final product.

This is where something like an AI Usage Scale becomes useful. Rather than attaching one AI policy to an entire course, teachers can make the expectations visible assignment by assignment.

Maybe the Product Is No Longer Enough

Generative AI exposes a weakness that existed long before ChatGPT. Educators have often inferred learning from products. A student submits an essay, so we assume the student can write an essay. A student turns in a lab report, so we assume the student understands the experiment. A student produces a presentation, so we assume the student understands what appears on the slides. Most of the time, those assumptions were reasonably safe. Today, they are becoming less safe.

That does not mean essays, reports, projects, or presentations have lost their value (in fact, I would argue that they have become even more important.) But it may mean that the finished product needs to become one piece of evidence among several.

Coming Next

Future editions of The AI Chronicles will continue examining assessment from several angles, including:

authentic assessment and oral defense; designing assignments around process rather than product; AI literacy as an assessable skill; what AI detection can and cannot tell us; the role of handwritten and in-class work; and what students themselves think meaningful assessment should look like.

Missed our discussion on NPR last month? Catch it below!