Debra M. Hardison
Ph.D. Linguistics, Cognitive Psychology; M.A. French Linguistics, Indiana University
Affiliation: Department of Linguistics, Languages, & Cultures, Michigan State University
Email: hardiso2@msu.edu
Multimodal Speech Research
My research program has focused on multimodal integration of information in spoken language processing, co-speech gesture, second-language speech production, and the applications of technology in perception and production training of the segmental and suprasegmental aspects of language. Some current projects involve:
the relationships among elements of second-language learners’ socioaffective profiles, language use factors, and interlocutor type in the development of the component skills of their oral interaction abilities, merging quantitative findings and qualitative data from participants’ learning profiles.
the coordination of gesture type, pitch, and segmental duration by Japanese language teachers.
the relationship between learners’ oral communication strategies in the second language and their speaking abilities.
Forthcoming book: The Multimodal Context of Phonological Learning. Equinox.
This book fills a need in applied linguistics by providing a comprehensive view of the multimodal context in which speech is perceived, produced, taught, and learned. It combines research review, original data from eye-movement research, and a pedagogical orientation. The book reviews the early foundation established by speechreading studies that explored the extraction of speech information from a talker's facial movements and the interaction of auditory-visual cues in perceptual illusions. It then focuses on the role of auditory, visual, and tactile information in second-language perceptual learning and the interactional functions of eye gaze within the complex of nonverbal communication that incorporates head movements and manual gestures. The book details an original mixed-methods study of the eye-gaze behavior of perceivers when viewing the face of a speaker producing their first language (English) and a speaker producing their second language (French) in different stimulus conditions. The issue of variability in speech emerges as a central theme throughout the book and one which researchers and teachers are encouraged to exploit through their selection of multimodal materials and their approach to pronunciation teaching.
With support from a Creating Inclusive Excellence grant, Maritza Medina Gonzalez, the Resource Center for Persons with Disabilities, and I developed ADEPT: Assistive Design for English Phonetic Tools. ADEPT is a combination of 3D tactile International Phonetic Alphabet symbol cards and a companion website designed for collaborative language learning between sighted and nonsighted users. Click here for a recorded introduction to the production of American English consonants and vowels with examples of each used in the study. Click here for the full story of the project.
This study won the Dorothy Chun Award for Best Journal Article in Language Learning & Technology in 2022. Click here for the story.