Turning AI into an Ally: Adapting Assessment for a Post-COVID, AI-Assisted ESL Writing Classroom

By: Ildi Porter-Szucs

Before the COVID-19 pandemic, my ESL students’ writing was evaluated through a combination of hand-written in-class essays and take-home essays, consisting of multiple drafts. However, the introduction of ChatGPT and other AI tools has forced a change. We now know, as highlighted by a 2022 New York Times investigation, that ChatGPT can generate writing that ranges from flawless to intentionally flawed, making it challenging to determine whether the work is genuinely student-produced. With no reliable way to detect unauthorized AI use, I developed a new assessment approach that recruits AI as personal tutor for the students rather than an adversary that I can only lose to.

Step 1: Diagnostic Writing Baseline Each unit begins with a hand-written, timed, in-class diagnostic. This assessment provides a baseline of students' unassisted writing on academic topics, which might include climate change, language and identity, business ethics, or the analysis of an art form. I evaluate this work both on a class-specific rubric and using the Common European Framework of Reference (CEFR), a reputable language proficiency framework, highlighting each student’s individual strengths and areas for improvement, such as organization, mechanics, or formulaic sequences.

Step 2: Guided AI Interaction Next, students type up their in-class writing, including all the errors, and then use ChatGPT to make corrections. [Before the first assignment, we spend class time learning how to use ChatGPT for legitimate purposes. For any students who don’t yet have an account, we create one together. We then practice giving the chatbot various commands and copy-pasting the results into a Word document.] The first command is to correct grammatical errors; the second is to rewrite at their target CEFR level. By comparing their original writing to AI's corrections, students identify patterns in their errors and compile these learning targets into their personal checklist. This self-analysis deepens their metacognitive skills, helping them see error types and frequency, rather than treating each mistake in isolation. During this stage, I support students in articulating specific learning targets, such as mastering irregular verbs or improving cohesion.

Step 3: Submission and Instructor Feedback Students submit their original and chatbot-revised work, along with their checklists, on Canvas. I review whether they are using ChatGPT responsibly, analyzing their errors accurately, and setting realistic learning targets. This ongoing assessment allows me to tailor in-class instruction to address common needs identified in students’ self-assessments.

Step 4: Building Skills Throughout the unit, we focus on topic-related content knowledge, vocabulary, grammar, and core writing skills like paraphrasing, summarizing, and integrating visual data. This phase of the work remains “old school”—without AI—ensuring that students develop essential skills. Once they have accomplished the goals set for the unit, students revise their writing, combining their original drafts with newly acquired knowledge. This intermediate draft is again assessed on completion rather than quality.

Step 5: Final Draft and AI-Assisted Revision Students then input this revised draft into ChatGPT for another round of grammar and CEFR-targeted revisions, updating their checklists based on new feedback. As they meet specific goals, they adjust or add new learning targets to their individual checklists.

Step 6: Post-Test and Instructor Reflection Finally, students complete a posttest by re-creating their revised essay from memory, by hand, in class, applying all they have learned. This writing takes the place of the former multi-draft take-home essay and is graded once again using both class-specific and CEFR rating scales. As part of my practice of the scholarship of teaching and learning, I evaluate the effectiveness of this new method by comparing pre- and post-tests on entirely new topics to gauge students' ability to transfer skills like cohesive device use and effective paragraph development to unfamiliar content.

In summary, teaching students proper chatbot use and using the chatbot for its strength in providing students with immediate and personalized corrections on their writing have allowed me to focus more on helping students to develop metacognitive skills, monitor their own learning, and write more than in the pre-AI model. Although I am only in my second semester of using this approach, the initial results are promising, as I see improved writing from first draft to first draft. As an added benefit, rather than battling AI, my students and I have recruited it as a personal tutor to aid in the learning process. 

I welcome your questions, comments, or suggestions on this approach.