Prepare for the Next Step
Introduction
In my early years of language teaching, I viewed assessment as merely being tests that take place at the end of the semester to inform students with a grade. Even though I intuitively knew that this was a fallible operationalization of tests, or at least a limited use of them, I did not have the required expertise to do otherwise.
In the Language Teaching Studies MA program, I was introduced to different types of assessment, of which each has a unique purpose. Most importantly, I was introduced to the principles that guide successful assessment design: reliability (by which we ensure the test/test administration is consistent across time, test-takers, and the items that constitute the test itself), validity (by which we verify the test aligns with its objective), authenticity (by which we make sure the test mirror real-world usage), and washback (by which we gauge the test to have a positive impact on future learning) (Brown & Abeywickrama, 2019).
The following artifacts demonstrate my growth in designing assessments grounded in these principles. At the same time, I believe there is always room for improvement. Just as assessment is an ongoing process, so too is my development as an assessment designer.
Artifact #1: Assessment creation for grammar – (LT 631: Measuring Language Ability)
I created this assessment example for LT 631: Measuring Language Ability. I designed this assessment task for grade 8 students enrolled in a private institution that adopts the Common Core State Standards (CCSS). I chose the CCSS standard: L.8.1.B Form and use verbs in the active and passive voice to be the objective of this assessment task. I wanted to create an assessment task that not only measures grammatical ability to form sentences in the active and passive voice, but also measures students’ knowledge of the nuance each voice offers. That said, the assessment task requirements are threefold: identifying the voice of each sentence in a given context, determining the agent/focus of the sentence, and transforming the sentences to the opposite voice. This way, the task measures the form-meaning link of the language being assessed. Thus, students become aware that the choice of voice is not only a matter of grammatical accuracy but also a way of conveying particular meanings and nuances. Larsen-Freeman (2015) defines grammar as having to do not only with form and meaning, but also with nuances of meaning when two or more grammatical structures can be used to express the same semantic meaning. This concept undergirds my assessment here as I embed the analysis of the forms and invite students to identify the nuanced message intended by active and passive voice.
I am convinced that adopting a communicative approach should not only show during instruction, but also should show in assessment. That way we establish the significance of the communicative purpose to language and align instruction, assessment and real world use of language.
Artifact #2: Assessment statistics interpretation– (LT 631: Measuring Language Ability)
In this artifact, I was given a dataset for listening, writing, and vocabulary exam results and other data that present the number of hours spent on playing videogames by the test-takers. My task was to calculate statistics of the listening exam such as item facility (IF), item discriminability (ID) and correlations between the listening exam results and the results of the other aforementioned exams and the hours spent on playing videogames. Interpreting this data was the most informative part as I could see how numbers and quantitative analysis of exam results could indicate insights about the validity and reliability of an exam. For example, if the split test correlations are high, it means that the exam has high reliability (consistency) because it means test-takers performed similarly on the two halves of the exam. Hughes (2003) explains that split test method is one of many methods that provide a coefficient of internal consistency by testing the correlation between the scores of two halves of a test in order to obtain the reliability coefficient. Also, if the item facility value is too high, it means that it probably is not measuring the intended construct which threatens the reliability of the exam. Brown & Abeywickrama, (2019) define item facility as the extent to which a test item is easy or difficult for the test-takers. They also explain that if a test is too easy or too difficult, the test does not do the work it is intended to do.
In the beginning of LT 631: Measuring Language Ability course, I learned that we will be doing statistics related to assessment. I was not sure of my ability in conducting statistical analysis and dealing with numbers. I remember repeating to my classmates, “I don’t have a mathematical background and I am not sure if I can do this”. In fact, when we dug deeper into it, it turned out to be the most enjoyable part of the course. Being able to analyze test results and use numbers to objectively determine the extent to which a test successfully fulfills its intended purpose reassured me that I do not have to rely solely on intuition when judging an exam’s reliability and validity. Although judgement calls and intuition are necessary, now I know that I can support those judgments with fairly objective evidence.
These insights were extremely useful and I can see myself using these quantitative methods in the future when designing tests.
Artifact #3: Assessment Plan – (LT 630: Curriculum and Materials Design)
“Assessment plays three interrelated and overlapping roles in course design. The first is assessing needs, the second is assessing students’ learning, and the third is evaluating the course itself” (Graves, 2000, p.207). Drawing on this concept, this artifact presents my assessment plan for the EFL course I created for LT 630: Curriculum and Material Design. From the three roles of assessment according to Graves (2000), my assessment plan focused on the second and third.
The process of creating this assessment plan required careful thinking about how to balance formative and summative assessment: evaluating students’ progress within the process of learning to guide them through developing their skills and measuring their achievement of the learning objectives at the end of each unit or at the end of the course (Brown & Abeywickrama, 2019). As a result, my assessment choices range between this and that until I came to deciding on how to assess listening, speaking, reading and writing. Given that the course focuses heavily on these four skills, I wanted a systematic summative way to assess them after each unit rather than relying solely on formative assessment.
Devising a plan for summative assessment for each of the four skills separately was a logistical challenge, given that each unit takes only four sessions to teach and there would not be enough time to allocate for assessing each skill separately before proceeding to the next unit. That was a perfect opportunity to utilize Integrated Performance assessments (IPAs): “a multitask assessment which is framed within a single thematic context [where] students first complete an interpretive task, then use the information learned in an interpersonal task, and finally summarize their learning with a presentational task.” (Adair-Hauck et al., 2006, p. 365).
Moreover, my assessment plan included course evaluation. I placed different measures for course evaluation at almost equal intervals during the course. These ranged between informal measures such as exit tickets at the end of each lesson, and more systemic ones such as mid-course survey and end-of-course survey. Also, the students’ learning assessments were also considered a formative part of course evaluation as they inform on whether the learning objectives were successfully met, which is the main indicator of the course’s success.
Adair-Hauck, B., Glisan, E. W., Koda, K., Swender, E. B., & Sandrock, P. (2006). The integrated performance assessment (IPA): Connecting assessment to instruction and learning. Foreign Language Annals, 39(3), pp.359–382. https://doi.org/10.1111/j.1944-9720.2006.tb02894.x
Brown, H. D., & Abeywickrama, P. (2019). Language assessment: Principles and classroom practices (3rd ed.). Pearson Education.
Graves, K. (2000). Designing language courses : a guide for teachers. Heinle & Heinle.
Hughes, A. (2003). Testing for language teachers (2nd edition.). Cambridge University Press.
Larsen-Freeman, D. (2015). Research into practice: grammar learning and teaching. Language Teaching, 48(2), pp.263–280. https://doi.org/10.1017/S0261444814000408