LangLearn
Language Learning Development
Shared Task at EVALITA 2023
Overview
Language Learning Development (LangLearn) is proposed at EVALITA as the first shared task on automatic language development assessment. The task consists of predicting the relative order of two essays written by the same student.
News
08/05/2023 - Test data sent to partipants
07/02/2023 - Training data are available in Data
10/11/2022 - We are online!
Motivation
Over the last twenty years, there has been a growing interest in exploiting the potential of Natural Language Processing (NLP) tools and machine learning methods in the context of language development, both in first (L1) and second language (L2) acquisition scenarios.
These systems have been devised with the aim of characterizing the properties of learners' language and how it evolves over time, across modalities and stages of acquisition. A similar concern has been paid to turning theoretical considerations into educational applications. It is the case of Intelligent Computer-Assisted Language Learning (ICALL) systems (Granger, 2003) and tools for automatically scoring learners’ writing with respect to language proficiency and writing quality (McNamara et al., 2015), and more generally systems able to automatically assign a learner's language production to a given developmental level (Sagae et al., 2005; Lu, 2009) or to operationalize sophisticated metrics of language development thus alleviating the laborious manual computation of these metrics by experts (Bram et al., 2014; Crossley et al., 2014; Lubetich et al., 2014).
Generally, a greater amount of studies has been carried out in the field of L2 learning where the study of L2 writings is seen as a proxy for language ability development (Crossley, 2020). In this respect, much related work is devoted to predicting the degree of L2 proficiency according to expert-based evaluation (Crossley et al., 2012) or to modelling the evolution of grammatical structures' competence with respect to predefined grades, e.g. the Common European Framework of Reference for Languages (CEFRL) (Vajjala et al., 2014; Volodina et al., 2016; Zilio et al., 2018). On the other side, fewer studies have focused on exploiting NLP techniques in the context of L1 development. Among these, we can mention studies devoted to assessing syntactic development in preschool children (Lubetich et al., 2014; Sagae, 2021) and to examine overall writing ability and its development during later language acquisition (Crossley et al., 2011).
It is worth noting that the majority of research on language development both in L1 and L2 has been focused on English. Few exceptions are represented by e.g. the works by (Weiss et al.,2019) and (Kerz etal., 2020), which investigated writing development in German-speaking students across the elementary and secondary school and (Miaschi et al., 2021; Miaschi et al., 2020), who proposed a methodology for tracking the evolution of written language competence of L1 Italian and L2 Spanish learners, respectively.
Contact the organizers: langlearn.evalita2023@gmail.com