Dudley, A., Marsden, E., and Bovolenta, G. (2024). The Context-Aligned Two Thousand Test: A new test of French high-frequency vocabulary size for beginner-to-low intermediate proficiency adolescent learners. Language Testing, 41(4), 759-791. https://doi.org/10.1177/02655322241261415
ABSTRACT
Vocabulary knowledge strongly predicts second language reading, listening, writing, and speaking. Yet, few tests have been developed to assess vocabulary knowledge in French. The primary aim of this pilot study was to design and initially validate the Context-Aligned Two Thousand Test (CA-TTT), following open research practices. The CA-TTT is a test of written form–meaning recognition of high-frequency vocabulary aimed at beginner-to-low intermediate learners of French at the end of their fifth year of secondary education. Using an argument-based validation framework, we drew on classical test theory and Rasch modeling, together with correlations with another vocabulary size test and proficiency measures, to assess the CA-TTT’s internal and external validity. Overall, the CA-TTT showed high internal and external validity. Our study highlighted the decisive role of the curriculum in determining vocabulary knowledge in instructed, low-exposure contexts. We discuss how this might contribute to under- or over-estimations of vocabulary size, depending on the relations between the test and curriculum content. Further research using the tool is openly invited, particularly with lower proficiency learners in this context. Following further validation, the test could serve as a tool for assessing high-frequency vocabulary knowledge at beginner-to-low intermediate levels, with due attention paid to alignment with curriculum content.
Dudley, A. & Marsden, E. (2024). The lexical content of high-stakes national exams in French, German, and Spanish in England. Foreign Language Annals, 57(2), pp.311-338.
ABSTRACT
Surprisingly little is known about the number and frequency level of words that beginner-to-low-intermediate 16-year-old learners of French, German, and Spanish are expected to know when taking high-stakes national exams in England. This study presents exploratory analyses of the lexical content of the listening and reading tests of these exams, a corpus totalling 116,645 running words. Specifically, it seeks to understand the number and frequency level of words that (a) this demographic seems to be expected to know and (b) could be needed for awarding organisations to create exams year-on-year. Key findings include that the proportion of low(er)-frequency words in the corpus of exam papers seemed large, given the stage of the learners and the purpose of the assessments. Critically, these low(er)-frequency words changed at a high rate between papers, likely incurring a heavy reliance on the lexical inferencing abilities of these relatively inexperienced language learners.
Marsden, E., Dudley, A., & Hawkes, R. (2023). Use of word lists in a high-stakes, low-exposure context: Topic-driven or frequency-informed. Modern Language Journal, 107(3), pp.669–692. https://doi.org/10.1111/modl.12866
ABSTRACT
The awarding organizations that create and administer high-stakes assessments for beginner-to-low-intermediate 16-year-old learners of French, German, and Spanish in England provide optional topic-driven word lists as guides for teachers and textbook writers. Given that these lists are developed by the awarding organizations, they exert a powerful washback effect on teaching and learning. However, we do not know how much of these lists have actually been used in exams. We therefore analyzed the extent to which these lists have been used when developing the General Certificate of Secondary Education listening and reading exams, a corpus totaling 116,647 words. One key finding showed that approximately half of the awarding organizations’ lists had never been used in any of the exams to date. Given recent changes to curriculum policy, we also investigated how word list type—frequency-informed versus the awarding organizations’ topic-driven lists—affected lexical coverage of the exams. Overall, our findings suggested that using the topic-driven lists was likely to be a suboptimal use of lesson time, as they did not provide learners with enough words to understand any given text with ease. Frequency-informed word lists, however, seemed to better prepare learners for the exams.
Dudley, A. and Marsden, E. (under review). Components of vocabulary competence and their relationships with listening and reading among beginner-to-low intermediate learners of French.
Pre-print | OSF repository
ABSTRACT
The extent to which vocabulary competence consists of interrelated components of lexical knowledge has been widely discussed. Yet, few studies have explored the extent to which lexical knowledge and lexical processing are psychometrically distinct or their relative contribution to listening and reading, particularly among beginner-to-low-intermediate learners of languages other than English. The current study examines the extent to which lexical knowledge and processing are (a) psychometrically distinct and (b) predict listening and reading. 218 beginner-to-low intermediate adolescent learners of French completed a battery of tests, yielding measures of high-frequency lexical knowledge (meaning recognition and form recognition and recall), lexical processing (speed and automaticity of word recognition), and listening and reading proficiency. Results suggested that at this stage of proficiency, lexical knowledge and processing were psychometrically distinct. Lexical knowledge (but not processing) was found to strongly predict listening and reading. Findings underscore the importance of high-frequency vocabulary at these early stages.
Dudley, A., and Marsden, E. (under contract for publication in Spring 2025). Components of Language Proficiency. Multilingual Matters.
ABSTRACT
Research (e.g., Jeon & In’nami, 2022) has largely shown that second language (L2) proficiency is a multicomponential construct in so far as it encompasses many linguistic and non-linguistic variables, including knowledge (e.g., sound-spelling correspondences, vocabulary, and grammar), skills (e.g., listening, reading, writing, and speaking), and affective factors (e.g., motivation and attitudes towards language learning). This body of research has largely focused on (a) how one component (e.g., vocabulary knowledge) predicts another component (e.g., reading skills) and (b) highly-educated L2 learners of English in university settings. For instance, each of the four meta-analyses presented in Jeon and In’nami (2022) reported English as the most widely studied L2 by far. As such, very few studies have explored the interaction between or within individual components of language proficiency—be it linguistic or non-linguistic—with the same group of L2 learners, not least with learners of languages other than English, such as French or Spanish, in low-exposure, highly-instructed (‘foreign’ language) contexts.
In England, this specific population of learners typically only receives 400-450 hours of exposure before sitting their high-stakes, national GCSE exams in the fifth year of secondary education, with very little—if any—exposure to the target language outside the classroom. These factors, together with the global dominance of English, are very likely to influence the nature of linguistic knowledge acquired and, in turn, the relations between proficiency and its linguistic and non-linguistic (affective) components. Likewise, our understanding of the state of linguistic knowledge among this population is extremely limited, with only a handful of available studies. It is thus difficult to know the extent to which findings from previous research on L2 English are generalisable within this educational context and similar low-exposure, highly-instructed, non-English contexts.
Moreover, this lack of knowledge has consequently meant that the language education community in England—that is, teachers, teacher educators, textbook publishers, exam writers, curriculum designers, policy makers, and researchers—have had to operate with a very limited understanding of these learners’ proficiency when developing learning, teaching, and testing materials, resources, and tools. A case in point being the recent changes in the high-stakes General Certificate in Secondary Education (GCSE) exams taken by over 250,000 16-year-olds every year in England.
The proposed book aims to address these gaps, by presenting findings from a comprehensive research study that investigated the (linguistic and non-linguistic) components of language proficiency among pre-intermediate 16-year-old learners of French and Spanish immediately after they completed their GCSE exams. Specifically, this book addresses three main research questions:
(1) What is the current state of linguistic knowledge among these learners? In particular, to what extent do the sub-components of each component of language knowledge interact? And to what extent does this vary as a function of the target language?
(2) To what extent do linguistic components (i.e., sound-spelling correspondences, vocabulary, grammar) predict L2 proficiency in the four skills (i.e., listening, reading, speaking, writing)? And to what extent does this vary as a function of the target language?
(3) To what extent do (a) components of linguistic knowledge (i.e., sound-spelling correspondences, vocabulary, grammar) and (b) proficiency (i.e., listening, reading, speaking, writing) relate to L2 motivation?