Generative AI and Anecdotal SoTL

By: Matt J Schumann

When I was invited to write a post on ChatGPT, I considered revising my presentation on student motivations from the New Beginnings conference in August. Now, nearing the end of a semester teaching about 100 students across five classes in three disciplines (History, Political Science, Academic Writing), I remain interested in student motivations, but my thesis has changed: echoing Ernest Boyer’s missive from 1990, faculty urgently need to turn their training and talents for research toward their classrooms.

Boyer’s “scholarship of teaching,” what we now call Scholarship of Teaching and Learning (SoTL), takes the questions that scholars ask in their respective academic disciplines and applies them to how faculty teach, how students learn, and some tools and technologies in between. While SoTL now uses a range of data to inform best practices, a more anecdotal, situational, idiosyncratic approach may suffice: we could certainly inform our teaching from a series of large studies, but we might also apply our own individual skills and instincts as researchers to the students we actually teach!

What do they say about generative AI? Many say nothing, but among those who use it and speak about it, it’s not just ChatGPT: they also use QuillBot, Scribbr, and Grammarly. Their use is uneven and highly situational, yet all seem to focus on the instruction they want, need, and are or are not receiving as they transition between k12 and college.

I’ll begin with QuillBot, which seems the simplest and most directly on a road to academic dishonesty and poor student learning.

Apparently developed with the noble intention of rephrasing or summarizing hard readings, I see my lower achievers using it most—and I thank them for admitting this—to rephrase from their sources, paste in their assignments, and thus avoid the challenge of research-writing from scratch.

Plagiarism seems to be a secondary concern, next to a lack of confidence and perhaps instruction about the other purpose of paraphrasing, summarizing, and similar essential practices in English Language Arts: wrestling with a text in order to understand it. Since my higher achievers tend to wrestle, rephrase, reframe, etc., almost instinctively, they generally do not use QuillBot.

They do, however, use Scribbr as an easy-access citation machine. As my classes typically range from 9th grade (honors and dual-enrollment) through the 200-level, my assignments may contain students’ first contact with the practice of citation. I suspect that most of us, at their stage, received our analog citation education as a kind of academic hazing, and that we, like my students, now prefer to click “cite” links when available instead of typing our citations keystroke by keystroke.

I often see students using Scribbr along these lines, helping them adapt to this arcane yet necessary practice. Even then, certainly in writing-intensive classes, my students’ most frequent question remains: “Dr. Schumann, how do I cite this?”

Then comes Grammarly, which I believe to be the most common AI, again due to lack of instruction and lack of confidence.

Whether in practice, or in feedback when I ask for it, my students consistently want to “write like adults,” “write more formally,” or just not make mistakes, while also using elevated vocabulary and complex grammar. Some of my highest achievers showed me how to type directly into Grammarly and let the AI function like an essay-level autocorrect feature.

In the background of this practice, I feel like I hear Rousseau, Piaget, and others like them shouting that these are developmental issues. Faculty cannot address this with rules or syllabus language, nor with punitive measures and assuming academic dishonesty. Instead, humanizing our students, we should try our best to discover and then address their actual motivations, insecurities, and educational lacunae.

As I describe this humanization, let me return to anecdotal SoTL. As we adapt to disruptive technologies, we must consult our inclinations and priorities as instructors, and dig deeply into our students’ drives, anxieties, and learning gaps. These data probably represent our best bet for adapting to generative AI and other new tools.

Incidentally, the assignments for which students used these programs involved essays. To counter them, we might use Turn-it-in or Copyleaks, but we could also direct them to other media. They can dress up their slides in Canva or Slidesgo, or present a screencast on Loom or Bandicam.

So now I come to ChatGPT, the only one of the AI tools that I would use myself. True to my presentation at New Beginnings, I have instructed my students how to use it—as a digital study-buddy, comparing against other sources like Wikipedia, Google Scholar and JSTOR—and how to cite it. In response…

• Those who received my instruction imperfectly used the technology imperfectly, yet predictably. They cited “ChatGPT” without noting the conversation title or their prompts, and often copy-pasted uncritically. They also used other sources poorly, indicating bigger problems than misuse of AI.

• Those who received my instruction well, however, used and cited ChatGPT adroitly. A U.S. History student (9th grade) played with counterfactuals, another in American Government (10th grade) ran abstract scenarios, and several in World History (11th grade) have used it in the best Quillbot style: trying to bring scholarship on esoteric subjects from the deep past down to their reading level.

What have I learned? Personally, I continue to struggle with how much instruction to give up-front, and yet…

• Especially with ChatGPT, I see value in using the technology and setting guidelines from my experience.

• Even more, I see value in walking with students through the technologies’ best uses as I understand them.

• Finally, in the best spirit of anecdotal SoTL, I see the most value in asking students how and why they use generative AI, and how to develop my teaching to better address their felt needs.