Measurement of raters' listening disfluency (LD)

1. Learners' shadowing
   Shadowing  is a special kind of  "repeat-after-me" practice, where a learner repeats a given audio with as a short delay as possible while  listening. Shadowing has been introduced to enhance learners' listening skills. In this project, shadowing is used for another purpose, where shadowing disfluency is quantitatively measured to indicate learners' listening disfluency (LD).

2. Raters' shadowing
   If we ask a rater to shadow a learner's read-aloud speech, what does the rater's listening disfluency (LD) indicate? Since our measurement of a rater's LD displays which parts of the learner's speech are intelligible or unintelligible to the rater, it quantifies the degree of (un)intelligibility of the read-aloud speech as sequential data. Since shadowing disfluencies are involuntary behaviors of raters, our method is regarded as objective and quantitative measurement of (un)intelligibility.

For details, please read the papers listed in References.

References

• Inoue, Y., Kabashima, S., Saito, D., Minematsu, N., Kanamura, K., Yamauchi, Y. (2018) A Study of Objective Measurement of Comprehensibility through Native Speakers' Shadowing of Learners' Utterances. Proc. Interspeech 2018, 1651-1655

• S. Kabashima, Y. Inoue, D. Saito and N. Minematsu, "DNN-Based Scoring of Language Learners’ Proficiency Using Learners’ Shadowings and Native Listeners’ Responsive Shadowings," 2018 IEEE Spoken Language Technology Workshop (SLT), Athens, Greece, 2018, pp. 971-978

• S. Ando et al., "A Large Collection of Sentences Read Aloud by Vietnamese Learners of Japanese and Native Speaker's Reverse Shadowings," 2019 22nd Conference of the Oriental COCOSDA International Committee for the Co-ordination and Standardisation of Speech Databases and Assessment Techniques (O-COCOSDA), Cebu, Philippines, 2019, pp. 1-6

• Lin, Z., Takashima, R., Saito, D., Minematsu, N., Nakanishi, N. (2020) Shadowability Annotation with Fine Granularity on L2 Utterances and its Improvement with Native Listeners’ Script-Shadowing. Proc. Interspeech 2020, 3865-3869

• C. Zhu, R. Hakoda, D. Saito, N. Minematsu, N. Nakanishi and T. Nishimura, "Multi-Granularity Annotation of Instantaneous Intelligibility of Learners' Utterances Based on Shadowing Techniques," 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), Cartagena, Colombia, 2021, pp. 1071-1078