In an effort to provide a theoretical rationale for the use of scenario-based assessment to measure competency-based, situated second and foreign (S/FL) proficiency, this paper traces conceptualizations of S/FL proficiency since the 1960s along with the major approaches to measuring these conceptualizations. The distinguishing characteristic of this evolution is that, as the construct broadened, the more complex the assessment methods became and the greater the potential for meaningful interpretation. This paper argues that while language-based, independent and integrated skill-based, and task-based approaches to S/FL proficiency assessment can be useful in certain assessment contexts, they are not engineered to measure S/FL proficiency in contexts of situated language use – that is, where goal-oriented task accomplishment is located within a sociocultural context, and where the ability to achieve complex tasks is embedded within the mediated engagements and social practices of a particular community. For this reason, some researchers have turned to scenario-based assessment (SBA). This paper describes how SBA has been used in the mainstream and S/FL assessment contexts, highlighting the affordances of this approach. Finally, the paper illustrates how a learning-oriented approach to assessment (LOA) (Purpura & Turner, 2018) can serve as a comprehensive conceptual assessment framework for engineering and validating SBAs.

Learning-oriented assessment (LOA) has gained considerable attention in second and foreign language (L2) research over the past two decades due to its close connection to learning and teaching. Greatly influenced by the theoretical and pedagogical foundations laid by classroom-based assess-ment (CBA) research and practices, LOA in classroom settings has been found to encompass several types of CBA with a similar learning-centered emphasis, such as “formative assessment,” “assessment for learning,” and “dynamic assessment” (Leung, 2020). While an abundance of research has examined how assessment can facilitate learning and teaching in L2 classrooms, Turner and Purpura (2016) pointed out that there has been little systematicity with regards to the approaches to pursuing classroom-based LOA research, most likely due to the complexity of the dynamics between learning, teaching, and assessment in L2 contexts.

The vast development of digital technology and the widespread use of social network platforms have reshaped how we live in the world. For L2 learners to maximally utilize their language proficiency to function effectively as members of modern society, they need not only the necessary L2 knowledge, skills, and abilities (KSAs) but also essential topical knowledge. While many researchers believe that topical knowledge should be viewed as an integral component of L2 communicative competence, the role of topical knowledge has not always been accounted for in an assessment context due to the difficulty of operationalizing the construct.

Scenario-based assessment, an innovative, technology-based assessment approach, allows great affordances for expanding the measured constructs of an assessment. It is designed expressly for learners to demonstrate their KSAs in a context that simulates real-life language use. Through the utilization of a sequence of thematically-related tasks, along with simulated character interaction, scenario-based assessment offers opportunities to examine L2 learners’ communicative competence in a purposeful, interactive, and contextually meaningful manner.

In this study, a scenario-based language assessment (SBLA) was developed to measure high-intermediate L2 learners’ topical knowledge and their L2 KSAs as part of the broadened construct of L2 communicative competence. To fulfill the scenario goal, learners were required to demonstrate their listening, reading, and writing abilities to build and share knowledge. In addition, learners’ prior topical knowledge was measured and their topical learning was tracked using the same set of topical knowledge items.

A total of 118 adult EFL learners participated in the study. The results showed that the SBLA served as an appropriate measure of high-intermediate learners’ L2 proficiency. The topical knowledge items were found to function appropriately, supporting the use of the SBLA to measure topical knowledge as part of the broadened construct of communicative competence. In addition, most learners exhibited substantial topical learning over the course of the SBLA, suggesting that with proper contextualization, learning can be facilitated within an assessment. In sum, this study demonstrated the potential value of scenario-based assessment as an approach to measure complex constructs of communicative language competence in L2 contexts.

To effectively achieve communicative goals, second language (L2) learners need not only the necessary L2 knowledge, skills, and abilities, but also the relevant topical knowledge. However, in L2 assessment contexts, very few existing tests-in-operation have examined L2 learners’ topical knowledge as part of their language performance. This study investigates the construct of topical knowledge in a scenario-based language assessment (SBLA) that simulates real-life language use of building and sharing knowledge. 41 L2 learners at the intermediate to advanced levels from an adult ESL program participated in the study. Topical knowledge was operationalized as content knowledge and lexical knowledge associated with the theme of the SBLA. Descriptive statistics, correlations, Rasch analysis, and repeated measures ANOVAs were used to examine the construct validity of topical knowledge in the SBLA. The results show that L2 learners’ topical knowledge encompasses both content and lexical knowledge, but they function differently in the process of building and sharing knowledge. Nevertheless, both play a role in the performance outcomes and should be recognized as an integral component of L2 proficiency. The findings also lend evidentiary support for the use of a highly-contextualized SBLA to broaden our understanding of the underlying construct of L2 communicative ability.

When most people think of language assessments, they generally recall the standardized formats of high-stakes, large-scale examinations. These assessments are widely recognized for providing valid and reliable measures of test-taker knowledge. However, recently these standardized assessments have also been criticized for not providing accurate reflections of the knowledge, skills, and abilities essential for 21^st century students to confront the era of digital communication. In order to meet these demands, assessment researchers and test designers have been re-conceptualizing test design and development procedures to provide more authentic assessments which more accurately reflect the complexity of tasks test-takers are likely to encounter in the 21^st century (Bachman & Palmer, 1996; Chapelle & Douglas, 2006; Purpura, 2004).

In second language (L2) testing literature, from the skills-and-elements approach to the more recent models of communicative language ability, the conceptualization of L2 proficiency has evolved and broadened over the past few decades (Purpura, 2016). Consequently, the notion of L2 speaking ability has also gone through change, which has influenced L2 testers to constantly reevaluate what needs to be assessed and how L2 speaking assessment can adopt different designs and techniques accordingly.

What comes to mind when the word assessment is mentioned for language classrooms? How is it to be interpreted? For many years, it has mainly been viewed as a tool to record student achievement through the types of items and tasks employed in traditional large-scale testing. In reality, however, much more happens in classrooms in terms of using assessment to support learning and inform teaching. With the growing awareness of assessment activities internal to the classroom and managed by teachers, classroom assessment has become an emerging paradigm of its own with a focus on learning and an evolving research agenda (Turner 2012). This chapter underscores this paradigm and concentrates on the local context of classrooms, where assessment has the potential to serve the learning process. Specifically, the focus is on second and foreign language (henceforth, L2) classrooms, where it has been claimed that assessment can serve as the “bridge” between teaching and learning (Colby-Kelly and Turner 2007). This culture in L2 classrooms, where assessment is central, has been referred to as learning-oriented assessment (LOA) (Purpura 2004, 2009; and more recently Purpura and Turner, forthcoming). The LOA approach is not to be confused with nor is it in competition with other current L2 classroom assessment techniques (e.g., diagnostic, Alderson 2005; dynamic, Lantolf and Poehner 2011), but certainly shares common characteristics with them. Its premise is to begin with learning, that is, to prioritize learning when considering the interrelationships across instruction, assessment and learning. Although research has moved in the direction of exploring these relationships, no study to date has “proposed that the discussion begin with an examination of the learning process” followed by considerations of how assessment can serve to enhance this process (Purpura 2004, 2009). This is the central theme on which LOA is based. A related challenge is to identify and determine the ways in which teachers and learners can make use of assessments and capitalize on the generated information to guide and support the learning process.

In this chapter we propose an initial framework in which to provide a way forward to explore the following questions: What are the characteristics of assessment instances in the classroom (planned and unplanned) that appear to provide a context for learning, and how is this supported by a theory of learning? What is the evidence that indicates that these interventions bring about change in learning as viewed through changes in proficiency? What is the nature of teacher/student and student/student feedback and interaction that influences the effort towards an outcome of further learning? We do this by first situating the emergence of LOA in an interdisciplinary, historical setting, which focuses on context, learning theory, and proficiency. We then discuss current research within this context. Next, we build on the research to date and present a working framework, which serves to explain as well as deconstruct LOA. Finally, we examine key issues that affect LOA and provide our view of future directions, both in practice and research.