To measure the quality of learners' spoken production (as a proxy for speaking proficiency), we asked learners to complete the following four tasks.
The role play prompts directed participants to talk to their French (or Spanish) friend (i.e., the research assistant) about their holiday. Six bullet points were laid out in the order in which they had to speak. Four prompted a response to a question from their interlocutor and two prompted a question. These prompts were provided during the preparation period.
Links to the materials
Instructions for examiners
Participants were asked to describe (question 1) and compare (question 2) two pictures (which they had seen during the preparation period). The first picture featured a mountain scene and the second a city scene. Participants then answered three questions (which they had not seen during the preparation period) about differences between living in the mountains and the city.
Links to the materials
Instructions for examiners
Participant read aloud a short passage (approximately 50 words in length and seen during the preparation period). The read-aloud was followed by an unprepared conversation about the passage—the dataset used for the current chapter. The French text discussed how the Internet has influenced our lives, and the Spanish text focused on Las Fallas, a cultural celebration. Participants answered four questions about the text.
Links to the materials
Instructions for examiners
Audio recordings from the above tasks were rated in terms of accentedness, comprehensibility, and fluency. We provide below a definition of each construct.
Accentedness: The degree to which the speech differed from a production pattern expected of a native speaker, with “heavy accent” describing speech that departs heavily from a native speaker’s production and is influenced by English and/or potentially other first languages and “no accent” referring to native-like production.
Comprehensibility: The degree of effort required by the listener to understand the speaker
Fluency: The pace of speech, with fluent performance characterised by few pauses and hesitations and an optimal (not too slow and not too fast) speaking rate.
Written transcripts of the audio recordings from the above tasks were also rated in terms of content and the accuracy and range of vocabulary and grammar used. See here for the scoring criteria.