HANDWRITTEN VERSUS TYPED NOTES:

THE IMPACT OF NOTE-TAKING MODES IN SECOND LANGUAGE LISTENING TESTS

The full dissertation is available at this link.

Abstract

Lecture comprehension is a critical academic skill for both first language (L1) and second language (L2) speakers, often accompanied by note-taking.  With the increasing portability of computers and their growing integration into education, technological advancements have influenced note-taking practices in classrooms as well as their consideration in standardized language assessments. Some tests, such as IELTS administered on paper and the TOEFL iBT, permit handwritten notes, while IELTS administered on computer allows typed notes. Tests like TOEIC and the Duolingo English Test prohibit note-taking altogether. However, these policies have largely been implemented based on practicality or security concerns, without considering empirical evidence related to impacts on listening comprehension. This study investigates the effects of different note-taking modes on listening test performance and the content of notes.

A total of 305 L1-Korean L2-English adults were recruited to take a listening test and randomly assigned to one of three note-taking conditions: handwriting (n = 102), typing (n = 102), or no note-taking (n = 101). Participants completed four listening testlets from official research forms of the TOEFL iBT; each testlet consisted of a five-minute lecture followed by six multiple-choice questions. Listening test performance was analyzed using linear regression at the test level and uniform and non-uniform Differential Item Functioning (DIF) analyses at the item level. Handwritten and typed notes were compared on measures of word count, information units, verbatim writing, translanguaging, and nonlinguistic elements using Mann-Whitney U tests.

The results indicated no significant differences in overall test scores between note-taking modes. Uniform DIF analyses revealed that one item requiring inference favored handwriting, while another item focusing on main ideas favored typing. No DIF was observed when comparing no note-taking with handwriting or typing. Non-uniform DIF emerged only among lower-ability test takers, with the same items flagged in uniform DIF analyses showing discrepancies between handwriting and typing.

Analysis of note content showed no significant differences between handwritten and typed notes in terms of word count and translanguaging. However, handwritten notes included more information units from the passage, verbatim text from the passage, and nonlinguistic elements than typed notes. Based on regression analyses, for the handwriting group, information units were the only significant predictor of test scores, while for the typing group, information units positively predicted scores and verbatim writing and translanguaging had negative impacts.

Although listening test scores did not differ significantly across note-taking modes, the findings suggest that listening processes may vary depending on the note-taking mode, as reflected in the note content. These results call for test developers to reassess their note-taking policies, considering the extent to which test scores accurately reflect the intended construct, align with the target language use domain, promote positive educational consequences, and ensure a pleasant test-taker experience while ensuring fair treatment of test takers. While varied concerns influence decisions about note-taking policies in standardized listening tests, empirical evidence on how note-taking might (not) impact test scores and test-taking processes should not be ignored.